Sample records for temporal reward structures
-
Yang, Xin-hua; Huang, Jia; Lan, Yong; Zhu, Cui-ying; Liu, Xiao-qun; Wang, Ye-fei; Cheung, Eric F C; Xie, Guang-rong; Chan, Raymond C K
2016-01-04
Anhedonia, the loss of interest or pleasure in reward processing, is a hallmark feature of major depressive disorder (MDD), but its underlying neurobiological mechanism is largely unknown. The present study aimed to examine the underlying neural mechanism of reward-related decision-making in patients with MDD. We examined behavioral and neural responses to rewards in patients with first-episode MDD (N=25) and healthy controls (N=25) using the Effort-Expenditure for Rewards Task (EEfRT). The task involved choices about possible rewards of varying magnitude and probability. We tested the hypothesis that individuals with MDD would exhibit a reduced neural response in reward-related brain structures involved in cost-benefit decision-making. Compared with healthy controls, patients with MDD showed significantly weaker responses in the left caudate nucleus when contrasting the 'high reward'-'low reward' condition, and blunted responses in the left superior temporal gyrus and the right caudate nucleus when contrasting high and low probabilities. In addition, hard tasks chosen during high probability trials were negatively correlated with superior temporal gyrus activity in MDD patients, while the same choices were negatively correlated with caudate nucleus activity in healthy controls. These results indicate that reduced caudate nucleus and superior temporal gyrus activation may underpin abnormal cost-benefit decision-making in MDD. Copyright © 2015 Elsevier Inc. All rights reserved.
-
Cavanagh, Sean E; Wallis, Joni D; Kennerley, Steven W; Hunt, Laurence T
2016-01-01
Correlates of value are routinely observed in the prefrontal cortex (PFC) during reward-guided decision making. In previous work (Hunt et al., 2015), we argued that PFC correlates of chosen value are a consequence of varying rates of a dynamical evidence accumulation process. Yet within PFC, there is substantial variability in chosen value correlates across individual neurons. Here we show that this variability is explained by neurons having different temporal receptive fields of integration, indexed by examining neuronal spike rate autocorrelation structure whilst at rest. We find that neurons with protracted resting temporal receptive fields exhibit stronger chosen value correlates during choice. Within orbitofrontal cortex, these neurons also sustain coding of chosen value from choice through the delivery of reward, providing a potential neural mechanism for maintaining predictions and updating stored values during learning. These findings reveal that within PFC, variability in temporal specialisation across neurons predicts involvement in specific decision-making computations. DOI: http://dx.doi.org/10.7554/eLife.18937.001 PMID:27705742
-
2014-09-29
Framing Reinforcement Learning from Human Reward: Reward Positivity, Temporal Discounting, Episodicity , and Performance W. Bradley Knox...positive a trainer’s reward values are; temporal discounting, the extent to which future reward is discounted in value; episodicity , whether task...learning occurs in discrete learning episodes instead of one continuing session; and task performance, the agent’s performance on the task the trainer
-
Neural Correlates of Temporal Credit Assignment in the Parietal Lobe
Eisenberg, Ian; Gottlieb, Jacqueline
2014-01-01
Empirical studies of decision making have typically assumed that value learning is governed by time, such that a reward prediction error arising at a specific time triggers temporally-discounted learning for all preceding actions. However, in natural behavior, goals must be acquired through multiple actions, and each action can have different significance for the final outcome. As is recognized in computational research, carrying out multi-step actions requires the use of credit assignment mechanisms that focus learning on specific steps, but little is known about the neural correlates of these mechanisms. To investigate this question we recorded neurons in the monkey lateral intraparietal area (LIP) during a serial decision task where two consecutive eye movement decisions led to a final reward. The underlying decision trees were structured such that the two decisions had different relationships with the final reward, and the optimal strategy was to learn based on the final reward at one of the steps (the “F” step) but ignore changes in this reward at the remaining step (the “I” step). In two distinct contexts, the F step was either the first or the second in the sequence, controlling for effects of temporal discounting. We show that LIP neurons had the strongest value learning and strongest post-decision responses during the transition after the F step regardless of the serial position of this step. Thus, the neurons encode correlates of temporal credit assignment mechanisms that allocate learning to specific steps independently of temporal discounting. PMID:24523935
-
High monetary reward rates and caloric rewards decrease temporal persistence
Bode, Stefan; Murawski, Carsten
2017-01-01
Temporal persistence refers to an individual's capacity to wait for future rewards, while forgoing possible alternatives. This requires a trade-off between the potential value of delayed rewards and opportunity costs, and is relevant to many real-world decisions, such as dieting. Theoretical models have previously suggested that high monetary reward rates, or positive energy balance, may result in decreased temporal persistence. In our study, 50 fasted participants engaged in a temporal persistence task, incentivised with monetary rewards. In alternating blocks of this task, rewards were delivered at delays drawn randomly from distributions with either a lower or higher maximum reward rate. During some blocks participants received either a caloric drink or water. We used survival analysis to estimate participants' probability of quitting conditional on the delay distribution and the consumed liquid. Participants had a higher probability of quitting in blocks with the higher reward rate. Furthermore, participants who consumed the caloric drink had a higher probability of quitting than those who consumed water. Our results support the predictions from the theoretical models, and importantly, suggest that both higher monetary reward rates and physiologically relevant rewards can decrease temporal persistence, which is a crucial determinant for survival in many species. PMID:28228517
-
High monetary reward rates and caloric rewards decrease temporal persistence.
Fung, Bowen J; Bode, Stefan; Murawski, Carsten
2017-02-22
Temporal persistence refers to an individual's capacity to wait for future rewards, while forgoing possible alternatives. This requires a trade-off between the potential value of delayed rewards and opportunity costs, and is relevant to many real-world decisions, such as dieting. Theoretical models have previously suggested that high monetary reward rates, or positive energy balance, may result in decreased temporal persistence. In our study, 50 fasted participants engaged in a temporal persistence task, incentivised with monetary rewards. In alternating blocks of this task, rewards were delivered at delays drawn randomly from distributions with either a lower or higher maximum reward rate. During some blocks participants received either a caloric drink or water. We used survival analysis to estimate participants' probability of quitting conditional on the delay distribution and the consumed liquid. Participants had a higher probability of quitting in blocks with the higher reward rate. Furthermore, participants who consumed the caloric drink had a higher probability of quitting than those who consumed water. Our results support the predictions from the theoretical models, and importantly, suggest that both higher monetary reward rates and physiologically relevant rewards can decrease temporal persistence, which is a crucial determinant for survival in many species. © 2017 The Authors.
-
Temporal Discounting and Inter-Temporal Choice in Rhesus Monkeys
Hwang, Jaewon; Kim, Soyoun; Lee, Daeyeol
2009-01-01
Humans and animals are more likely to take an action leading to an immediate reward than actions with delayed rewards of similar magnitudes. Although such devaluation of delayed rewards has been almost universally described by hyperbolic discount functions, the rate of this temporal discounting varies substantially among different animal species. This might be in part due to the differences in how the information about reward is presented to decision makers. In previous animal studies, reward delays or magnitudes were gradually adjusted across trials, so the animals learned the properties of future rewards from the rewards they waited for and consumed previously. In contrast, verbal cues have been used commonly in human studies. In the present study, rhesus monkeys were trained in a novel inter-temporal choice task in which the magnitude and delay of reward were indicated symbolically using visual cues and varied randomly across trials. We found that monkeys could extract the information about reward delays from visual symbols regardless of the number of symbols used to indicate the delay. The rate of temporal discounting observed in the present study was comparable to the previous estimates in other mammals, and the animal's choice behavior was largely consistent with hyperbolic discounting. Our results also suggest that the rate of temporal discounting might be influenced by contextual factors, such as the novelty of the task. The flexibility furnished by this new inter-temporal choice task might be useful for future neurobiological investigations on inter-temporal choice in non-human primates. PMID:19562091
-
Model-free and model-based reward prediction errors in EEG.
Sambrook, Thomas D; Hardwick, Ben; Wills, Andy J; Goslin, Jeremy
2018-05-24
Learning theorists posit two reinforcement learning systems: model-free and model-based. Model-based learning incorporates knowledge about structure and contingencies in the world to assign candidate actions with an expected value. Model-free learning is ignorant of the world's structure; instead, actions hold a value based on prior reinforcement, with this value updated by expectancy violation in the form of a reward prediction error. Because they use such different learning mechanisms, it has been previously assumed that model-based and model-free learning are computationally dissociated in the brain. However, recent fMRI evidence suggests that the brain may compute reward prediction errors to both model-free and model-based estimates of value, signalling the possibility that these systems interact. Because of its poor temporal resolution, fMRI risks confounding reward prediction errors with other feedback-related neural activity. In the present study, EEG was used to show the presence of both model-based and model-free reward prediction errors and their place in a temporal sequence of events including state prediction errors and action value updates. This demonstration of model-based prediction errors questions a long-held assumption that model-free and model-based learning are dissociated in the brain. Copyright © 2018 Elsevier Inc. All rights reserved.
-
Common and distinct neural correlates of personal and vicarious reward: A quantitative meta-analysis
Morelli, Sylvia A.; Sacchet, Matthew D.; Zaki, Jamil
2015-01-01
Individuals experience reward not only when directly receiving positive outcomes (e.g., food or money), but also when observing others receive such outcomes. This latter phenomenon, known as vicarious reward, is a perennial topic of interest among psychologists and economists. More recently, neuroscientists have begun exploring the neuroanatomy underlying vicarious reward. Here we present a quantitative whole-brain meta-analysis of this emerging literature. We identified 25 functional neuroimaging studies that included contrasts between vicarious reward and a neutral control, and subjected these contrasts to an activation likelihood estimate (ALE) meta-analysis. This analysis revealed a consistent pattern of activation across studies, spanning structures typically associated with the computation of value (especially ventromedial prefrontal cortex) and mentalizing (including dorsomedial prefrontal cortex and superior temporal sulcus). We further quantitatively compared this activation pattern to activation foci from a previous meta-analysis of personal reward. Conjunction analyses yielded overlapping VMPFC activity in response to personal and vicarious reward. Contrast analyses identified preferential engagement of the nucleus accumbens in response to personal as compared to vicarious reward, and in mentalizing-related structures in response to vicarious as compared to personal reward. These data shed light on the common and unique components of the reward that individuals experience directly and through their social connections. PMID:25554428
-
Reward alters the perception of time.
Failing, Michel; Theeuwes, Jan
2016-03-01
Recent findings indicate that monetary rewards have a powerful effect on cognitive performance. In order to maximize overall gain, the prospect of earning reward biases visual attention to specific locations or stimulus features improving perceptual sensitivity and processing. The question we addressed in this study is whether the prospect of reward also affects the subjective perception of time. Here, participants performed a prospective timing task using temporal oddballs. The results show that temporal oddballs, displayed for varying durations, presented in a sequence of standard stimuli were perceived to last longer when they signaled a relatively high reward compared to when they signaled no or low reward. When instead of the oddball the standards signaled reward, the perception of the temporal oddball remained unaffected. We argue that by signaling reward, a stimulus becomes subjectively more salient thereby modulating its attentional deployment and distorting how it is perceived in time. Copyright © 2015 Elsevier B.V. All rights reserved.
-
Influence of reward motivation on human declarative memory.
Miendlarzewska, Ewa A; Bavelier, Daphne; Schwartz, Sophie
2016-02-01
Motivational relevance can prioritize information for memory encoding and consolidation based on reward value. In this review, we pinpoint the possible psychological and neural mechanisms by which reward promotes learning, from guiding attention to enhancing memory consolidation. We then discuss how reward value can spill-over from one conditioned stimulus to a non-conditioned stimulus. Such generalization can occur across perceptually similar items or through more complex relations, such as associative or logical inferences. Existing evidence suggests that the neurotransmitter dopamine boosts the formation of declarative memory for rewarded information and may also control the generalization of reward values. In particular, temporally-correlated activity in the hippocampus and in regions of the dopaminergic circuit may mediate value-based decisions and facilitate cross-item integration. Given the importance of generalization in learning, our review points to the need to study not only how reward affects later memory but how learned reward values may generalize to related representations and ultimately alter memory structure. Copyright © 2015 Elsevier Ltd. All rights reserved.
-
Distinct medial temporal networks encode surprise during motivation by reward versus punishment
Murty, Vishnu P.; LaBar, Kevin S.; Adcock, R. Alison
2016-01-01
Adaptive motivated behavior requires predictive internal representations of the environment, and surprising events are indications for encoding new representations of the environment. The medial temporal lobe memory system, including the hippocampus and surrounding cortex, encodes surprising events and is influenced by motivational state. Because behavior reflects the goals of an individual, we investigated whether motivational valence (i.e., pursuing rewards versus avoiding punishments) also impacts neural and mnemonic encoding of surprising events. During functional magnetic resonance imaging (fMRI), participants encountered perceptually unexpected events either during the pursuit of rewards or avoidance of punishments. Despite similar levels of motivation across groups, reward and punishment facilitated the processing of surprising events in different medial temporal lobe regions. Whereas during reward motivation, perceptual surprises enhanced activation in the hippocampus, during punishment motivation surprises instead enhanced activation in parahippocampal cortex. Further, we found that reward motivation facilitated hippocampal coupling with ventromedial PFC, whereas punishment motivation facilitated parahippocampal cortical coupling with orbitofrontal cortex. Behaviorally, post-scan testing revealed that reward, but not punishment, motivation resulted in greater memory selectivity for surprising events encountered during goal pursuit. Together these findings demonstrate that neuromodulatory systems engaged by anticipation of reward and punishment target separate components of the medial temporal lobe, modulating medial temporal lobe sensitivity and connectivity. Thus, reward and punishment motivation yield distinct neural contexts for learning, with distinct consequences for how surprises are incorporated into predictive mnemonic models of the environment. PMID:26854903
-
Distinct medial temporal networks encode surprise during motivation by reward versus punishment.
Murty, Vishnu P; LaBar, Kevin S; Adcock, R Alison
2016-10-01
Adaptive motivated behavior requires predictive internal representations of the environment, and surprising events are indications for encoding new representations of the environment. The medial temporal lobe memory system, including the hippocampus and surrounding cortex, encodes surprising events and is influenced by motivational state. Because behavior reflects the goals of an individual, we investigated whether motivational valence (i.e., pursuing rewards versus avoiding punishments) also impacts neural and mnemonic encoding of surprising events. During functional magnetic resonance imaging (fMRI), participants encountered perceptually unexpected events either during the pursuit of rewards or avoidance of punishments. Despite similar levels of motivation across groups, reward and punishment facilitated the processing of surprising events in different medial temporal lobe regions. Whereas during reward motivation, perceptual surprises enhanced activation in the hippocampus, during punishment motivation surprises instead enhanced activation in parahippocampal cortex. Further, we found that reward motivation facilitated hippocampal coupling with ventromedial PFC, whereas punishment motivation facilitated parahippocampal cortical coupling with orbitofrontal cortex. Behaviorally, post-scan testing revealed that reward, but not punishment, motivation resulted in greater memory selectivity for surprising events encountered during goal pursuit. Together these findings demonstrate that neuromodulatory systems engaged by anticipation of reward and punishment target separate components of the medial temporal lobe, modulating medial temporal lobe sensitivity and connectivity. Thus, reward and punishment motivation yield distinct neural contexts for learning, with distinct consequences for how surprises are incorporated into predictive mnemonic models of the environment. Copyright © 2016 Elsevier Inc. All rights reserved.
-
Cherniawsky, Avital S; Holroyd, Clay B
2013-03-01
Impulsivity is characterized in part by heightened sensitivity to immediate relative to future rewards. Although previous research has suggested that "high discounters" in intertemporal choice tasks tend to prefer immediate over future rewards because they devalue the latter, it remains possible that they instead overvalue immediate rewards. To investigate this question, we recorded the reward positivity, a component of the event-related brain potential (ERP) associated with reward processing, with participants engaged in a task in which they received both immediate and future rewards and nonrewards. The participants also completed a temporal discounting task without ERP recording. We found that immediate but not future rewards elicited the reward positivity. High discounters also produced larger reward positivities to immediate rewards than did low discounters, indicating that high discounters relatively overvalued immediate rewards. These findings suggest that high discounters may be more motivated than low discounters to work for monetary rewards, irrespective of the time of arrival of the incentives.
-
Temporal Discounting When the Choice Is Between Two Delayed Rewards
ERIC Educational Resources Information Center
Green, Leonard; Myerson, Joel; Macaux, Eric W.
2005-01-01
The present experiments extend the temporal discounting paradigm from choice between an immediate and a delayed reward to choice between 2 delayed rewards: a smaller amount of money available sooner and a larger amount available later. Across different amounts and delays, the data were consistently well described by a hyperbola-like discounting…
-
Gender Dimorphism of Brain Reward System Volumes in Alcoholism
Sawyer, Kayle S.; Oscar-Berman, Marlene; Barthelemy, Olivier J.; Papadimitriou, George M.; Harris, Gordon J.; Makris, Nikos
2017-01-01
The brain's reward network has been reported to be smaller in alcoholic men compared to nonalcoholic men, but little is known about the volumes of reward regions in alcoholic women. Morphometric analyses were performed on magnetic resonance brain scans of 60 long-term chronic alcoholics (ALC; 30 men) and 60 nonalcoholic controls (NC; 29 men). We derived volumes of total brain, and cortical and subcortical reward-related structures including the dorsolateral prefrontal (DLPFC), orbitofrontal, and cingulate cortices, and the temporal pole, insula, amygdala, hippocampus, nucleus accumbens septi (NAc), and ventral diencephalon (VDC). We examined the relationships of the volumetric findings to drinking history. Analyses revealed a significant gender interaction for the association between alcoholism and total reward network volumes, with ALC men having smaller reward volumes than NC men and ALC women having larger reward volumes than NC women. Analyses of a priori subregions revealed a similar pattern of reward volume differences with significant gender interactions for DLPFC and VDC. Overall, the volume of the cerebral ventricles in ALC participants was negatively associated with duration of abstinence, suggesting decline in atrophy over time. PMID:28285206
-
Pornpattananangkul, Narun; Nadig, Ajay; Heidinger, Storm; Walden, Keegan; Nusslock, Robin
2017-01-01
Although waiting for a reward reduces or discounts its value, some people have a stronger tendency to wait for larger rewards and forgo smaller-but-immediate rewards. This ability to delay gratification is captured by individual differences in so-called inter-temporal choices in which individuals are asked to choose between larger-but-delayed versus smaller-but-immediate rewards. The current study used event-related potentials (ERPs) to examine whether enhancement in two neuro-cognitive processes, outcome-anticipation and outcome-evaluation, modulate individual variability in inter-temporal responses. After completing a behavioral inter-temporal choice task, 34 participants performed an ERP gambling task. From this ERP task, we separately examined individual differences in outcome-anticipation (Stimulus-Preceding Negativity, SPN), early outcome-evaluation (Feedback-Related Negativity; FRN) and late outcome-evaluation (P3). We observed that both elevated outcome-anticipation (SPN) and late outcome-evaluation (P3) neural processes predicted a stronger preference toward larger-but-delayed rewards. No relationship was observed between inter-temporal responses and early outcome-evaluation (FRN), indicating that the relationship between outcome-evaluation and inter-temporal responses was specific to the late outcome-evaluation processing stream. Moreover, multiple regression analyses indicated that the SPN and P3 independently modulate individual differences in inter-temporal responses, suggesting separate mechanisms underlie the relationship between these two neuro-cognitive processes and inter-temporal responses. Accordingly, we identify two potential neural-cognitive modulators of individual variability in inter-temporal responses. We discuss the mechanisms underlying these modulators in terms of anticipation-related processing (SPN) and a saliency-bias toward gain (compared to loss) outcomes (P3). PMID:28224457
-
Neural dynamics of reward probability coding: a Magnetoencephalographic study in humans
Thomas, Julie; Vanni-Mercier, Giovanna; Dreher, Jean-Claude
2013-01-01
Prediction of future rewards and discrepancy between actual and expected outcomes (prediction error) are crucial signals for adaptive behavior. In humans, a number of fMRI studies demonstrated that reward probability modulates these two signals in a large brain network. Yet, the spatio-temporal dynamics underlying the neural coding of reward probability remains unknown. Here, using magnetoencephalography, we investigated the neural dynamics of prediction and reward prediction error computations while subjects learned to associate cues of slot machines with monetary rewards with different probabilities. We showed that event-related magnetic fields (ERFs) arising from the visual cortex coded the expected reward value 155 ms after the cue, demonstrating that reward value signals emerge early in the visual stream. Moreover, a prediction error was reflected in ERF peaking 300 ms after the rewarded outcome and showing decreasing amplitude with higher reward probability. This prediction error signal was generated in a network including the anterior and posterior cingulate cortex. These findings pinpoint the spatio-temporal characteristics underlying reward probability coding. Together, our results provide insights into the neural dynamics underlying the ability to learn probabilistic stimuli-reward contingencies. PMID:24302894
-
Gender dimorphism of brain reward system volumes in alcoholism.
Sawyer, Kayle S; Oscar-Berman, Marlene; Barthelemy, Olivier J; Papadimitriou, George M; Harris, Gordon J; Makris, Nikos
2017-05-30
The brain's reward network has been reported to be smaller in alcoholic men compared to nonalcoholic men, but little is known about the volumes of reward regions in alcoholic women. Morphometric analyses were performed on magnetic resonance brain scans of 60 long-term chronic alcoholics (ALC; 30 men) and 60 nonalcoholic controls (NC; 29 men). We derived volumes of total brain, and cortical and subcortical reward-related structures including the dorsolateral prefrontal (DLPFC), orbitofrontal, and cingulate cortices, and the temporal pole, insula, amygdala, hippocampus, nucleus accumbens septi (NAc), and ventral diencephalon (VDC). We examined the relationships of the volumetric findings to drinking history. Analyses revealed a significant gender interaction for the association between alcoholism and total reward network volumes, with ALC men having smaller reward volumes than NC men and ALC women having larger reward volumes than NC women. Analyses of a priori subregions revealed a similar pattern of reward volume differences with significant gender interactions for DLPFC and VDC. Overall, the volume of the cerebral ventricles in ALC participants was negatively associated with duration of abstinence, suggesting decline in atrophy with greater length of sobriety. Copyright © 2017 Elsevier Ireland Ltd. All rights reserved.
-
Steinglass, Joanna E; Lempert, Karolina M; Choo, Tse-Hwei; Kimeldorf, Marcia B; Wall, Melanie; Walsh, B Timothy; Fyer, Abby J; Schneier, Franklin R; Simpson, H Blair
2017-05-01
Temporal discounting refers to the tendency for rewards to lose value as the expected delay to receipt increases. Individuals with anorexia nervosa (AN) have been found to show reduced temporal discounting rates, indicating a greater preference for delayed rewards compared to healthy peers. Obsessive-compulsive disorder (OCD) and social anxiety disorder (SAD) commonly co-occur with AN, and anxiety has been related to development and prognosis of AN. We examined whether reduced temporal discounting is present across these potentially related disorders, and explored the relationship between temporal discounting and anxiety transdiagnostically. One hundred ninety six individuals (75 healthy controls (HC); 50 OCD; 27 AN; 44 SAD) completed two temporal discounting tasks in which they chose between smaller-sooner and larger-later monetary rewards. Two measures of discounting-discount rate and discount factor-were compared between diagnostic groups, and associations with anxious traits were examined. Individuals with AN showed decreased temporal discounting compared to HC. OCD and SAD groups did not differ significantly from HC. Across the sample, anxiety was associated with decreased discounting; more anxious individuals showed a greater preference for delayed reward. We replicated the findings that individuals with AN show an increased preference for delayed reward relative to HC and that individuals with OCD do not differ from HC. We also showed that individuals with SAD do not differ from HC in discounting. Across this large sample, two measures of anxious temperament were associated with temporal discounting. These data raise new questions about the relationship between this dimensional trait and psychopathology. © 2016 Wiley Periodicals, Inc.
-
Steinglass, Joanna E.; Lempert, Karolina M.; Choo, Tse-Hwei; Kimeldorf, Marcia B.; Wall, Melanie; Walsh, B. Timothy; Fyer, Abby J.; Schneier, Franklin R.; Simpson, H. Blair
2018-01-01
Background Temporal discounting refers to the tendency for rewards to lose value as the expected delay to receipt increases. Individuals with anorexia nervosa (AN) have been found to show reduced temporal discounting rates, indicating a greater preference for delayed rewards compared to healthy peers. Obsessive–compulsive disorder (OCD) and social anxiety disorder (SAD) commonly co-occur with AN, and anxiety has been related to development and prognosis of AN. We examined whether reduced temporal discounting is present across these potentially related disorders, and explored the relationship between temporal discounting and anxiety trans-diagnostically. Methods One hundred ninety six individuals (75 healthy controls (HC); 50 OCD; 27 AN; 44 SAD) completed two temporal discounting tasks in which they chose between smaller-sooner and larger-later monetary rewards. Two measures of discounting—discount rate and discount factor—were compared between diagnostic groups, and associations with anxious traits were examined. Results Individuals with AN showed decreased temporal discounting compared to HC. OCD and SAD groups did not differ significantly from HC. Across the sample, anxiety was associated with decreased discounting; more anxious individuals showed a greater preference for delayed reward. Conclusions We replicated the findings that individuals with AN show an increased preference for delayed reward relative to HC and that individuals with OCD do not differ from HC. We also showed that individuals with SAD do not differ from HC in discounting. Across this large sample, two measures of anxious temperament were associated with temporal discounting. These data raise new questions about the relationship between this dimensional trait and psychopathology. PMID:28009473
-
The date-delay framing effect in temporal discounting depends on substance abuse.
Klapproth, Florian
2012-07-01
In the present study, individuals with substance use disorders (n=30) and non-addicted controls (n=30) were presented with a delay-discounting task with time being described either as dates or as temporal intervals. Three main results were obtained. First, in both groups reward size had a large impact on discounting future rewards, with discount rates becoming larger with smaller reward sizes. Second, participants discounted future rewards less strongly when their time of delivery was presented as a date instead of a temporal distance. Third, whereas discount rates of individuals with substance use disorders varied substantially with regard to the presentation of time in the task, the controls changed their choices depending on time presentation only slightly. Copyright © 2012 Elsevier B.V. All rights reserved.
-
Goldstein, Rita Z.; Cottone, Lisa A.; Jia, Zhiru; Maloney, Thomas; Volkow, Nora D.; Squires, Nancy K.
2008-01-01
Temporal correlates of the brain circuit underlying reward processing in healthy adults remain unclear. The current study investigated the P3 and contingent negative variation (CNV) as putative reward-related temporal markers. The effect of sustained monetary reward on these event-related potentials and on behavior was assessed using a warned reaction-time paradigm in 16 young healthy subjects. Monetary reward (0, 1 and 45 cents) varied across blocks of trials. While the CNV was unaffected by money, P3 amplitude was significantly larger for 45 than the 1 and 0 cent conditions. This effect corresponded to the monotonically positive subjective ratings of interest and excitement on the task (45>1>0). These findings suggest a difference between the P3 and CNV; the P3 is sensitive to the sustained effect of relative reward value while the CNV does not vary with reward magnitude. PMID:16876894
-
A Mechanism for Reducing Delay Discounting by Altering Temporal Attention
Radu, Peter T; Yi, Richard; Bickel, Warren K; Gross, James J; McClure, Samuel M
2011-01-01
Rewards that are not immediately available are discounted compared to rewards that are immediately available. The more a person discounts a delayed reward, the more likely that person is to have a range of behavioral problems, including clinical disorders. This latter observation has motivated the search for interventions that reduce discounting. One surprisingly simple method to reduce discounting is an “explicit-zero” reframing that states default or null outcomes. Reframing a classical discounting choice as “something now but nothing later” versus “nothing now but more later” decreases discount rates. However, it is not clear how this “explicit-zero” framing intervention works. The present studies delineate and test two possible mechanisms to explain the phenomenon. One mechanism proposes that the explicit-zero framing creates the impression of an improving sequence, thereby enhancing the present value of the delayed reward. A second possible mechanism posits an increase in attention allocation to temporally distant reward representations. In four experiments, we distinguish between these two hypothesized mechanisms and conclude that the temporal attention hypothesis is superior for explaining our results. We propose a model of temporal attention whereby framing affects intertemporal preferences by modifying present bias. PMID:22084496
-
Enhanced Neural Responses to Imagined Primary Rewards Predict Reduced Monetary Temporal Discounting.
Hakimi, Shabnam; Hare, Todd A
2015-09-23
The pervasive tendency to discount the value of future rewards varies considerably across individuals and has important implications for health and well-being. Here, we used fMRI with human participants to examine whether an individual's neural representation of an imagined primary reward predicts the degree to which the value of delayed monetary payments is discounted. Because future rewards can never be experienced at the time of choice, imagining or simulating the benefits of a future reward may play a critical role in decisions between alternatives with either immediate or delayed benefits. We found that enhanced ventromedial prefrontal cortex response during imagined primary reward receipt was correlated with reduced discounting in a separate monetary intertemporal choice task. Furthermore, activity in enhanced ventromedial prefrontal cortex during reward imagination predicted temporal discounting behavior both between- and within-individual decision makers with 62% and 73% mean balanced accuracy, respectively. These results suggest that the quality of reward imagination may impact the degree to which future outcomes are discounted. Significance statement: We report a novel test of the hypothesis that an important factor influencing the discount rate for future rewards is the quality with which they are imagined or estimated in the present. Previous work has shown that temporal discounting is linked to individual characteristics ranging from general intelligence to the propensity for addiction. We demonstrate that individual differences in a neurobiological measure of primary reward imagination are significantly correlated with discounting rates for future monetary payments. Moreover, our neurobiological measure of imagination can be used to accurately predict choice behavior both between and within individuals. These results suggest that improving reward imagination may be a useful therapeutic target for individuals whose high discount rates promote detrimental behaviors. Copyright © 2015 the authors 0270-6474/15/3513103-07$15.00/0.
-
Temporal (In)Stability of Employee Preferences for Rewards
ERIC Educational Resources Information Center
Wine, Byron; Gilroy, Shawn; Hantula, Donald A.
2012-01-01
This study examined the temporal stability of employee preferences for rewards over seven monthly evaluations. Participants completed a ranking stimulus preference assessment monthly, and the latter six monthly assessments were compared to the initial assessment. Correlations of preferences from month to month ranged from r = -0.89 to 0.99.…
-
Functional requirements for reward-modulated spike-timing-dependent plasticity.
Frémaux, Nicolas; Sprekeler, Henning; Gerstner, Wulfram
2010-10-06
Recent experiments have shown that spike-timing-dependent plasticity is influenced by neuromodulation. We derive theoretical conditions for successful learning of reward-related behavior for a large class of learning rules where Hebbian synaptic plasticity is conditioned on a global modulatory factor signaling reward. We show that all learning rules in this class can be separated into a term that captures the covariance of neuronal firing and reward and a second term that presents the influence of unsupervised learning. The unsupervised term, which is, in general, detrimental for reward-based learning, can be suppressed if the neuromodulatory signal encodes the difference between the reward and the expected reward-but only if the expected reward is calculated for each task and stimulus separately. If several tasks are to be learned simultaneously, the nervous system needs an internal critic that is able to predict the expected reward for arbitrary stimuli. We show that, with a critic, reward-modulated spike-timing-dependent plasticity is capable of learning motor trajectories with a temporal resolution of tens of milliseconds. The relation to temporal difference learning, the relevance of block-based learning paradigms, and the limitations of learning with a critic are discussed.
-
Takahashi, Yuji K.; Langdon, Angela J.; Niv, Yael; Schoenbaum, Geoffrey
2016-01-01
Summary Dopamine neurons signal reward prediction errors. This requires accurate reward predictions. It has been suggested that the ventral striatum provides these predictions. Here we tested this hypothesis by recording from putative dopamine neurons in the VTA of rats performing a task in which prediction errors were induced by shifting reward timing or number. In controls, the neurons exhibited error signals in response to both manipulations. However, dopamine neurons in rats with ipsilateral ventral striatal lesions exhibited errors only to changes in number and failed to respond to changes in timing of reward. These results, supported by computational modeling, indicate that predictions about the temporal specificity and the number of expected rewards are dissociable, and that dopaminergic prediction-error signals rely on the ventral striatum for the former but not the latter. PMID:27292535
-
Frank, Guido K.; Shott, Megan E.; Hagman, Jennifer O.; Mittal, Vijay A.
2013-01-01
Objective The pathophysiology of the eating disorder anorexia nervosa remains obscure, but structural brain alterations could be functionally important biomarkers. Here we assessed taste pleasantness and reward sensitivity in relation to brain structure, which might be related to food avoidance commonly seen in eating disorders. Method We used structural magnetic resonance brain imaging to study gray and white matter volumes in individuals with restricting type currently ill (n = 19) or recovered-anorexia nervosa (n = 24), bulimia nervosa (n= 19) and healthy control women (n=24). Results All eating disorder groups showed increased gray matter volume of the medial orbitofrontal cortex (gyrus rectus). Manually tracing confirmed larger gyrus rectus volume, and predicted taste pleasantness across all groups. The analyses also indicated other morphological differences between diagnostic categories: Ill and recovered-anorexia nervosa had increased right, while bulimia nervosa had increased left antero-ventral insula gray matter volumes compared to controls. Furthermore, dorsal striatum volumes were reduced in recovered-anorexia and bulimia nervosa, and predicted sensitivity to reward in the eating disorder groups. The eating disorder groups also showed reduced white matter in right temporal and parietal areas when compared to healthy controls. Notably, the results held when controlling for a range of covariates (e.g., age, depression, anxiety, medications). Conclusion Brain structure in medial orbitofrontal cortex, insula and striatum is altered in eating disorders and suggests altered brain circuitry that has been associated with taste pleasantness and reward value. PMID:23680873
-
Fam, Justine; Westbrook, Fred; Arabzadeh, Ehsan
2016-01-01
Identifying similarities and differences in choice behavior across species is informative about how basic mechanisms give rise to more complex processes. In the present study, we compared pre- and post-choice latencies between rats and humans under two paradigms. In Experiment 1, we used a cued choice paradigm where subjects were presented with a cue that directed them as to which of two options to respond for rewards. In Experiment 2, subjects were free to choose between two options in order to procure rewards. In both Experiments rewards were delivered with distinct probabilities. The trial structure used in these experiments allowed the choice process to be decomposed into pre- and post-choice processes. Overall, post-choice latencies reflected the difference in reward probability between the two options, where latencies for the option with higher probability of reward were longer than those for the option with lower probability of reward. An interesting difference between rats and humans was observed: the choice behavior for humans, but not rats, was sensitive to the free-choice aspect of the tasks, such that in free-choice trials post-choice latencies no longer reflected the difference in reward probabilities between the two options. PMID:26862000
-
Hintsa, Taina; Hintsanen, Mirka; Jokela, Markus; Pulkki-Råback, Laura; Keltikangas-Järvinen, Liisa
2013-06-01
Personality dispositions may influence perceptions of work stress. The paper examines the relationship between temperament in terms of Strelau’s Regulative Theory of Temperament and the effort-reward imbalance and its components. There were 890 participants (360 men) aged 37.9 years on average. Temperament traits of briskness and perseveration (temporal characteristics of behavior), sensory sensitivity, emotional reactivity, endurance and activity (energetic characteristics of behavior) were measured by Strelau & Zawadzki’s Formal Characteristics of Behavior- Temperament Inventory (FCB-TI) in 1997 and 2001. Effort and reward at work were assessed with the original effort- reward imbalance (ERI) questionnaire of 2007. Higher ERI at work was predicted by higher emotional reactivity, higher perseveration, lower briskness, and lower endurance. Higher effort and lower rewards at work were predicted by higher perseveration and lower endurance. The FCB-TI temperament characteristics accounted for 5.2%, 4.8% and 6.5% of the variance in the ERI, effort and reward, respectively. Lower emotional reactivity, lower perseveration, higher brisk- ness and higher endurance predicted higher esteem at work, job promotion and job security. Individual differences in arousability, reflected in temporal and energetic characteristics of behavior, may predispose to or to protect from an effort-reward imbalance at work. Individual differences should be acknowledged in work stress prevention and developing interventions.
-
Temporal Discounting of Hypothetical Monetary Rewards by Adolescents, Adults, and Older Adults
ERIC Educational Resources Information Center
Whelan, Robert; McHugh, Louise A.
2009-01-01
The present experiment examined temporal discounting across 3 different age bands: adolescents, adults, and older adults (mean ages 14, 46, and 73 years, respectively). A computerized task was employed in which participants were asked to choose between larger rewards available at a specified time in the future--either 100 British Pounds or 1,000…
-
Kujawa, Autumn; Carroll, Ashley; Mumper, Emma; Mukherjee, Dahlia; Kessel, Ellen M; Olino, Thomas; Hajcak, Greg; Klein, Daniel N
2017-11-04
Brain regions involved in reward processing undergo developmental changes from childhood to adolescence, and alterations in reward-related brain function are thought to contribute to the development of psychopathology. Event-related potentials (ERPs), such as the reward positivity (RewP) component, are valid measures of reward responsiveness that are easily assessed across development and provide insight into temporal dynamics of reward processing. Little work has systematically examined developmental changes in ERPs sensitive to reward. In this longitudinal study of 75 youth assessed 3 times across 6years, we used principal components analyses (PCA) to differentiate ERPs sensitive to monetary reward and loss feedback in late childhood, early adolescence, and middle adolescence. We then tested reliability of, and developmental changes in, ERPs. A greater number of ERP components differentiated reward and loss feedback in late childhood compared to adolescence, but components in childhood accounted for only a small proportion of variance. A component consistent with RewP was the only one to consistently emerge at each of the 3 assessments. RewP demonstrated acceptable reliability, particularly from early to middle adolescence, though reliability estimates varied depending on scoring approach and developmental period. The magnitude of the RewP component did not significantly change across time. Results provide insight into developmental changes in the structure of ERPs sensitive to reward, and indicate that RewP is a consistently observed and relatively stable measure of reward responsiveness, particularly across adolescence. Copyright © 2017. Published by Elsevier B.V.
-
ERIC Educational Resources Information Center
Capaldi, E. J.; Martins, Ana; Miller, Ronald M.
2007-01-01
Rats in a Pavlovian situation were trained under three different reward schedules, at either a 30 s or a 90 s intertrial interval (ITI): Consistent reward (C), 50% irregular reward (I), and single alternation of reward and nonrewarded trials (SA). Activity was recorded to the conditioned stimulus (CS) and in all 10 s bins in each ITI except the…
-
Weak reward source memory in depression reflects blunted activation of VTA/SN and parahippocampus
Dobbins, Ian G.; Pizzagalli, Diego A.
2014-01-01
Reward responses in the medial temporal lobes and dopaminergic midbrain boost episodic memory formation in healthy adults, and weak memory for emotionally positive material in depression suggests this mechanism may be dysfunctional in major depressive disorder (MDD). To test this hypothesis, we performed a study in which unmedicated adults with MDD and healthy controls encoded drawings paired with reward or zero tokens during functional magnetic resonance imaging. In a recognition test, participants judged whether drawings were previously associated with the reward token (‘reward source’) or the zero token (‘zero source’). Unlike controls, depressed participants failed to show better memory for drawings from the reward source vs the zero source. Consistent with predictions, controls also showed a stronger encoding response to reward tokens vs zero tokens in the right parahippocampus and dopaminergic midbrain, whereas the MDD group showed the opposite pattern—stronger responses to zero vs reward tokens—in these regions. Differential activation of the dopaminergic midbrain by reward vs zero tokens was positively correlated with the reward source memory advantage in controls, but not depressed participants. These data suggest that weaker memory for positive material in depression reflects blunted encoding responses in the dopaminergic midbrain and medial temporal lobes. PMID:24078019
-
Schevernels, Hanne; Krebs, Ruth M.; Santens, Patrick; Woldorff, Marty G.; Boehler, C. Nico
2013-01-01
Recently, attempts have been made to disentangle the neural underpinnings of preparatory processes related to reward and attention. Functional magnetic resonance imaging (fMRI) research showed that neural activity related to the anticipation of reward and to attentional demands invokes neural activity patterns featuring large-scale overlap, along with some differences and interactions. Due to the limited temporal resolution of fMRI, however, the temporal dynamics of these processes remain unclear. Here, we report an event-related potentials (ERP) study in which cued attentional demands and reward prospect were combined in a factorial design. Results showed that reward prediction dominated early cue processing, as well as the early and later parts of the contingent negative variation (CNV) slow-wave ERP component that has been associated with task-preparation processes. Moreover these reward-related electrophysiological effects correlated across participants with response-time speeding on reward-prospect trials. In contrast, cued attentional demands affected only the later part of the CNV, with the highest amplitudes following cues predicting high-difficulty potential-reward targets, thus suggesting maximal task preparation when the task requires it and entails reward prospect. Consequently, we suggest that task-preparation processes triggered by reward can arise earlier, and potentially more directly, than strategic top-down aspects of preparation based on attentional demands. PMID:24064071
-
Humans Can Adopt Optimal Discounting Strategy under Real-Time Constraints
Schweighofer, N; Shishida, K; Han, C. E; Okamoto, Y; Tanaka, S. C; Yamawaki, S; Doya, K
2006-01-01
Critical to our many daily choices between larger delayed rewards, and smaller more immediate rewards, are the shape and the steepness of the function that discounts rewards with time. Although research in artificial intelligence favors exponential discounting in uncertain environments, studies with humans and animals have consistently shown hyperbolic discounting. We investigated how humans perform in a reward decision task with temporal constraints, in which each choice affects the time remaining for later trials, and in which the delays vary at each trial. We demonstrated that most of our subjects adopted exponential discounting in this experiment. Further, we confirmed analytically that exponential discounting, with a decay rate comparable to that used by our subjects, maximized the total reward gain in our task. Our results suggest that the particular shape and steepness of temporal discounting is determined by the task that the subject is facing, and question the notion of hyperbolic reward discounting as a universal principle. PMID:17096592
-
Lansink, Carien S; Meijer, Guido T; Lankelma, Jan V; Vinck, Martin A; Jackson, Jadin C; Pennartz, Cyriel M A
2016-10-12
The use of information from the hippocampal memory system in motivated behavior depends on its communication with the ventral striatum. When an animal encounters cues that signal subsequent reward, its reward expectancy is raised. It is unknown, however, how this process affects hippocampal dynamics and their influence on target structures, such as ventral striatum. We show that, in rats, reward-predictive cues result in enhanced hippocampal theta and beta band rhythmic activity during subsequent action, compared with uncued goal-directed navigation. The beta band component, also labeled theta's harmonic, involves selective hippocampal CA1 cell groups showing frequency doubling of firing periodicity relative to theta rhythmicity and it partitions the theta cycle into segments showing clear versus poor spike timing organization. We found that theta phase precession occurred over a wider range than previously reported. This was apparent from spikes emitted near the peak of the theta cycle exhibiting large "phase precessing jumps" relative to spikes in foregoing cycles. Neither this phenomenon nor the regular manifestation of theta phase precession was affected by reward expectancy. Ventral striatal neuronal firing phase-locked not only to hippocampal theta, but also to beta band activity. Both hippocampus and ventral striatum showed increased synchronization between neuronal firing and local field potential activity during cued compared with uncued goal approaches. These results suggest that cue-triggered reward expectancy intensifies hippocampal output to target structures, such as the ventral striatum, by which the hippocampus may gain prioritized access to systems modulating motivated behaviors. Here we show that temporally discrete cues raising reward expectancy enhance both theta and beta band activity in the hippocampus once goal-directed navigation has been initiated. These rhythmic activities are associated with increased synchronization of neuronal firing patterns in the hippocampus and the connected ventral striatum. When transmitted to downstream target structures, this expectancy-related state of intensified processing in the hippocampus may modulate goal-directed action. Copyright © 2016 the authors 0270-6474/16/3610598-13$15.00/0.
-
Emotional arousal predicts intertemporal choice
Lempert, Karolina M.; Johnson, Eli; Phelps, Elizabeth A.
2016-01-01
People generally prefer immediate rewards to rewards received after a delay, often even when the delayed reward is larger. This phenomenon is known as temporal discounting. It has been suggested that preferences for immediate rewards may be due to their being more concrete than delayed rewards. This concreteness may evoke an enhanced emotional response. Indeed, manipulating the representation of a future reward to make it more concrete has been shown to heighten the reward’s subjective emotional intensity, making people more likely to choose it. Here we use an objective measure of arousal – pupil dilation – to investigate if emotional arousal mediates the influence of delayed reward concreteness on choice. We recorded pupil dilation responses while participants made choices between immediate and delayed rewards. We manipulated concreteness through time interval framing: delayed rewards were presented either with the date on which they would be received (e.g., “$30, May 3”; DATE condition, more concrete) or in terms of delay to receipt (e.g., “$30, 7 days; DAYS condition, less concrete). Contrary to prior work, participants were not overall more patient in the DATE condition. However, there was individual variability in response to time framing, and this variability was predicted by differences in pupil dilation between conditions. Emotional arousal increased as the subjective value of delayed rewards increased, and predicted choice of the delayed reward on each trial. This study advances our understanding of the role of emotion in temporal discounting. PMID:26882337
-
Dopamine reward prediction-error signalling: a two-component response
Schultz, Wolfram
2017-01-01
Environmental stimuli and objects, including rewards, are often processed sequentially in the brain. Recent work suggests that the phasic dopamine reward prediction-error response follows a similar sequential pattern. An initial brief, unselective and highly sensitive increase in activity unspecifically detects a wide range of environmental stimuli, then quickly evolves into the main response component, which reflects subjective reward value and utility. This temporal evolution allows the dopamine reward prediction-error signal to optimally combine speed and accuracy. PMID:26865020
-
Wolosin, Sasha M.; Zeithamova, Dagmar; Preston, Alison R.
2012-01-01
Emerging evidence suggests that motivation enhances episodic memory formation through interactions between medial temporal lobe (MTL) structures and dopaminergic midbrain. In addition, recent theories propose that motivation specifically facilitates hippocampal associative binding processes, resulting in more detailed memories that are readily reinstated from partial input. Here, we used high-resolution functional magnetic resonance imaging to determine how motivation influences associative encoding and retrieval processes within human MTL subregions and dopaminergic midbrain. Participants intentionally encoded object associations under varying conditions of reward and performed a retrieval task during which studied associations were cued from partial input. Behaviorally, cued recall performance was superior for high-value relative to low-value associations; however, participants differed in the degree to which rewards influenced memory. The magnitude of behavioral reward modulation was associated with reward-related activation changes in dentate gyrus/CA2,3 during encoding and enhanced functional connectivity between dentate gyrus/CA2,3 and dopaminergic midbrain during both the encoding and retrieval phases of the task. These findings suggests that within the hippocampus, reward-based motivation specifically enhances dentate gyrus/CA2,3 associative encoding mechanisms through interactions with dopaminergic midbrain. Furthermore, within parahippocampal cortex and dopaminergic midbrain regions, activation associated with successful memory formation was modulated by reward across the group. During the retrieval phase, we also observed enhanced activation in hippocampus and dopaminergic midbrain for high-value associations that occurred in the absence of any explicit cues to reward. Collectively, these findings shed light on fundamental mechanisms through which reward impacts associative memory formation and retrieval through facilitation of MTL and VTA/SN processing. PMID:22524296
-
A Mechanism for Reducing Delay Discounting by Altering Temporal Attention
ERIC Educational Resources Information Center
Radu, Peter T.; Yi, Richard; Bickel, Warren K.; Gross, James J.; McClure, Samuel M.
2011-01-01
Rewards that are not immediately available are discounted compared to rewards that are immediately available. The more a person discounts a delayed reward, the more likely that person is to have a range of behavioral problems, including clinical disorders. This latter observation has motivated the search for ions that reduce discounting. One…
-
Reward Magnitude Effects on Temporal Discrimination
ERIC Educational Resources Information Center
Galtress, Tiffany; Kirkpatrick, Kimberly
2010-01-01
Changes in reward magnitude or value have been reported to produce effects on timing behavior, which have been attributed to changes in the speed of an internal pacemaker in some instances and to attentional factors in other cases. The present experiments therefore aimed to clarify the effects of reward magnitude on timing processes. In Experiment…
-
ERIC Educational Resources Information Center
Demurie, Ellen; Roeyers, Herbert; Baeyens, Dieter; Sonuga-Barke, Edmund
2013-01-01
It has been shown that delayed consumable rewards are discounted to a higher degree than money, which has been referred to as the "domain effect". Until now the effects of reward type on temporal discounting (TD) have mainly been studied in adults. Although there is evidence that children with attention-deficit/hyperactivity disorder (ADHD) tend…
-
Demurie, Ellen; Roeyers, Herbert; Baeyens, Dieter; Sonuga-Barke, Edmund
2012-11-01
It has been difficult to differentiate attention-deficit/hyperactivity disorder (ADHD) and autism spectrum disorder (ASD) in terms of some aspects of their cognitive profile. While both show deficits in executive functions, it has been suggested that they may differ in their response to monetary reward. For instance, children with ADHD prefer small immediate over large delayed rewards more than typically developing controls. One explanation for this is that they discount the value of rewards to a higher degree as they are moved into the future. The current study investigated whether children with ADHD can be differentiated from those with ASD in terms of reward discounting. Thirty-nine children (8-16 y) with ADHD, 34 children with ASD and 46 typically developing controls performed a hypothetical monetary temporal discounting task. Participants were instructed to make repeated choices between small variable rewards (0, 5, 10, 20, 30€) delivered immediately and large rewards delivered after a variable delay. Children with ADHD but not ASD discounted future rewards at a higher rate than typically developing controls. These data confirm steeper discounting of future rewards in ADHD and add to a small but growing literature showing that the psychological profile of ADHD can be distinguished from that of ASD in terms of disrupted motivational processes. © 2012 Blackwell Publishing Ltd.
-
NASA Astrophysics Data System (ADS)
Seymour, Ben; Barbe, Michael; Dayan, Peter; Shiner, Tamara; Dolan, Ray; Fink, Gereon R.
2016-09-01
Deep brain stimulation (DBS) of the subthalamic nucleus in Parkinson’s disease is known to cause a subtle but important adverse impact on behaviour, with impulsivity its most widely reported manifestation. However, precisely which computational components of the decision process are modulated is not fully understood. Here we probe a number of distinct subprocesses, including temporal discount, outcome utility, instrumental learning rate, instrumental outcome sensitivity, reward-loss trade-offs, and perseveration. We tested 22 Parkinson’s Disease patients both on and off subthalamic nucleus deep brain stimulation (STN-DBS), while they performed an instrumental learning task involving financial rewards and losses, and an inter-temporal choice task for financial rewards. We found that instrumental learning performance was significantly worse following stimulation, due to modulation of instrumental outcome sensitivity. Specifically, patients became less sensitive to decision values for both rewards and losses, but without any change to the learning rate or reward-loss trade-offs. However, we found no evidence that DBS modulated different components of temporal impulsivity. In conclusion, our results implicate the subthalamic nucleus in a modulation of outcome value in experience-based learning and decision-making in Parkinson’s disease, suggesting a more pervasive role of the subthalamic nucleus in the control of human decision-making than previously thought.
-
Seymour, Ben; Barbe, Michael; Dayan, Peter; Shiner, Tamara; Dolan, Ray; Fink, Gereon R.
2016-01-01
Deep brain stimulation (DBS) of the subthalamic nucleus in Parkinson’s disease is known to cause a subtle but important adverse impact on behaviour, with impulsivity its most widely reported manifestation. However, precisely which computational components of the decision process are modulated is not fully understood. Here we probe a number of distinct subprocesses, including temporal discount, outcome utility, instrumental learning rate, instrumental outcome sensitivity, reward-loss trade-offs, and perseveration. We tested 22 Parkinson’s Disease patients both on and off subthalamic nucleus deep brain stimulation (STN-DBS), while they performed an instrumental learning task involving financial rewards and losses, and an inter-temporal choice task for financial rewards. We found that instrumental learning performance was significantly worse following stimulation, due to modulation of instrumental outcome sensitivity. Specifically, patients became less sensitive to decision values for both rewards and losses, but without any change to the learning rate or reward-loss trade-offs. However, we found no evidence that DBS modulated different components of temporal impulsivity. In conclusion, our results implicate the subthalamic nucleus in a modulation of outcome value in experience-based learning and decision-making in Parkinson’s disease, suggesting a more pervasive role of the subthalamic nucleus in the control of human decision-making than previously thought. PMID:27624437
-
2018-01-01
Abstract Dopamine has been suggested to be crucially involved in effort-related choices. Key findings are that dopamine depletion (i) changed preference for a high-cost, large-reward option to a low-cost, small-reward option, (ii) but not when the large-reward option was also low-cost or the small-reward option gave no reward, (iii) while increasing the latency in all the cases but only transiently, and (iv) that antagonism of either dopamine D1 or D2 receptors also specifically impaired selection of the high-cost, large-reward option. The underlying neural circuit mechanisms remain unclear. Here we show that findings i–iii can be explained by the dopaminergic representation of temporal-difference reward-prediction error (TD-RPE), whose mechanisms have now become clarified, if (1) the synaptic strengths storing the values of actions mildly decay in time and (2) the obtained-reward-representing excitatory input to dopamine neurons increases after dopamine depletion. The former is potentially caused by background neural activity–induced weak synaptic plasticity, and the latter is assumed to occur through post-depletion increase of neural activity in the pedunculopontine nucleus, where neurons representing obtained reward exist and presumably send excitatory projections to dopamine neurons. We further show that finding iv, which is nontrivial given the suggested distinct functions of the D1 and D2 corticostriatal pathways, can also be explained if we additionally assume a proposed mechanism of TD-RPE calculation, in which the D1 and D2 pathways encode the values of actions with a temporal difference. These results suggest a possible circuit mechanism for the involvements of dopamine in effort-related choices and, simultaneously, provide implications for the mechanisms of TD-RPE calculation. PMID:29468191
-
Gratitude intervention modulates P3 amplitude in a temporal discounting task.
Patalano, Andrea L; Lolli, Sydney L; Sanislow, Charles A
2018-06-23
Gratitude has been shown to reduce economic impatience. In particular, individuals induced to experience heightened gratitude are more willing to choose delayed larger rewards over immediate smaller rewards (i.e., they have lower discounting rates) than those in a neutral condition. Using the event-related potential (ERP) method, we investigated the relation between gratitude level and neurophysiological correlates. Of interest was motivated information processing, as indexed by the P3 component. Participants were administered a gratitude or a neutral mood induction followed by a temporal discounting task (choosing between a fixed immediate reward versus a future reward that varied across trials) while electroencephalogram (EEG) activity was recorded. Individuals in the gratitude condition had greater P3 amplitude, suggesting greater attention to the future-reward option (the choice option that varied across trials), even when this option was not selected, and providing the first evidence of gratitude-induced changes in electrophysiological activity. Copyright © 2018. Published by Elsevier B.V.
-
Mesolimbic Dopamine Signals the Value of Work
Hamid, Arif A.; Pettibone, Jeffrey R.; Mabrouk, Omar S.; Hetrick, Vaughn L.; Schmidt, Robert; Vander Weele, Caitlin M.; Kennedy, Robert T.; Aragona, Brandon J.; Berke, Joshua D.
2015-01-01
Dopamine cell firing can encode errors in reward prediction, providing a learning signal to guide future behavior. Yet dopamine is also a key modulator of motivation, invigorating current behavior. Existing theories propose that fast (“phasic”) dopamine fluctuations support learning, while much slower (“tonic”) dopamine changes are involved in motivation. We examined dopamine release in the nucleus accumbens across multiple time scales, using complementary microdialysis and voltammetric methods during adaptive decision-making. We first show that minute-by-minute dopamine levels covary with reward rate and motivational vigor. We then show that second-by-second dopamine release encodes an estimate of temporally-discounted future reward (a value function). We demonstrate that changing dopamine immediately alters willingness to work, and reinforces preceding action choices by encoding temporal-difference reward prediction errors. Our results indicate that dopamine conveys a single, rapidly-evolving decision variable, the available reward for investment of effort, that is employed for both learning and motivational functions. PMID:26595651
-
Maximum likelihood methods for investigating reporting rates of rings on hunter-shot birds
Conroy, M.J.; Morgan, B.J.T.; North, P.M.
1985-01-01
It is well known that hunters do not report 100% of the rings that they find on shot birds. Reward studies can be used to estimate what this reporting rate is, by comparison of recoveries of rings offering a monetary reward, to ordinary rings. A reward study of American Black Ducks (Anas rubripes) is used to illustrate the design, and to motivate the development of statistical models for estimation and for testing hypotheses of temporal and geographic variation in reporting rates. The method involves indexing the data (recoveries) and parameters (reporting, harvest, and solicitation rates) by geographic and temporal strata. Estimates are obtained under unconstrained (e.g., allowing temporal variability in reporting rates) and constrained (e.g., constant reporting rates) models, and hypotheses are tested by likelihood ratio. A FORTRAN program, available from the author, is used to perform the computations.
-
Discounting of Delayed Food Rewards in Pigeons and Rats: Is There a Magnitude Effect
ERIC Educational Resources Information Center
Green, Leonard; Myerson, Joel; Holt, Daniel D.; Slavin, John R.; Estle, Sara J.
2004-01-01
Temporal discounting refers to the decrease in the present, subjective value of a reward as the time to its receipt increases. Results from humans have shown that a hyperbola-like function describes the form of the discounting function when choices involve hypothetical monetary rewards. In addition, magnitude effects have been reported in which…
-
Giordano, Louis A; Bickel, Warren K; Loewenstein, George; Jacobs, Eric A; Marsch, Lisa; Badger, Gary J
2002-09-01
A growing literature suggests that excessive temporal discounting of delayed rewards may be a contributing factor in the etiology of substance abuse problems. Little is known, however, about how drug deprivation may affect temporal discounting of delayed rewards by drug-dependent individuals. To examine the extent to which opioid deprivation affects how opioid-dependent individuals discount small, medium and large quantities of delayed heroin and money. Thirteen opioid-dependent individuals maintained on buprenorphine completed a hypothetical choice task in which they choose between a constant delayed reward amount and an immediate reward amount that was adjusted until they expressed indifference between both outcomes. The task was completed for three values of heroin and money rewards during eight sessions under conditions of opioid deprivation (four sessions) and satiation (four sessions). Across conditions, hyperbolic functions provided a good fit for the discounting data. Degree of discounting was significantly higher when subjects were opioid deprived. Consistent with previous findings, degree of discounting was higher for heroin than money and inversely related to the magnitude of the reward. Opioid deprivation increased the degree to which dependent individuals discounted delayed heroin and money. Understanding the conditions that affect how drug-dependent individuals discount delayed rewards might help us understand the myopic choices made by such individuals and help improve treatment outcomes.
-
de Water, Erik; Cillessen, Antonius H N; Scheres, Anouk
2014-01-01
Age-related differences in temporal discounting (TD) and risk taking, and their association, were examined in adolescents and young adults (n = 337) aged 12-27 years. Since monetary rewards are typically used in TD and risk-taking tasks, the association between monetary reward valuation and age and decision making in these tasks was explored as well. TD declined linearly with age, with a particularly sharp decline from 15 to 16 years. In contrast, risk taking was not correlated with age and TD. Reward valuation was not associated with TD and risk taking, and age-related differences in TD remained significant after controlling for reward valuation. Together, these findings suggest that risk taking and TD are two separate constructs with distinct age-related differences in adolescence and young adulthood. © 2014 The Authors. Child Development © 2014 Society for Research in Child Development, Inc.
-
Predictive Validity of Delay Discounting Behavior in Adolescence: A Longitudinal Twin Study
Isen, Joshua D.; Sparks, Jordan C.; Iacono, William G.
2014-01-01
A standard assumption in the delay discounting literature is that individuals who exhibit steeper discounting of hypothetical rewards also experience greater difficulty deferring gratification to real-world rewards. There is ample cross-sectional evidence that delay discounting paradigms reflect a variety of maladaptive psychosocial outcomes, including substance use pathology. We sought to determine whether a computerized assessment of hypothetical delay discounting (HDD) taps into behavioral impulsivity in a community sample of adolescent twins (N = 675). Using a longitudinal design, we hypothesized that greater HDD at age 14–15 predicts real-world impulsive choices and risk for substance use disorders in late adolescence. We also examined the genetic and environmental structure of HDD performance. Individual differences in HDD behavior showed moderate heritability, and were prospectively associated with real-world temporal discounting at age 17–18. Contrary to expectations, HDD was not consistently related to substance use or trait impulsivity. Although a significant association between HDD behavior and past substance use emerged in males, this effect was mediated by cognitive ability. In both sexes, HDD failed to predict a comprehensive index of substance use problems and behavioral disinhibition in late adolescence. In sum, we present some of the first evidence that HDD performance is heritable and predictive of real-world temporal discounting of rewards. Nevertheless, HDD might not serve as a valid marker of substance use disorder risk in younger adolescents, particularly females. PMID:24999868
-
Self-Control Assessments of Capuchin Monkeys With the Rotating Tray Task and the Accumulation Task
Beran, Michael J.; Perdue, Bonnie M.; Rossettie, Mattea S.; James, Brielle T.; Whitham, Will; Walker, Bradlyn; Futch, Sara E.; Parrish, Audrey E.
2016-01-01
Recent studies of delay of gratification in capuchin monkeys using a rotating tray (RT) task have shown improved self-control performance in these animals in comparison to the accumulation (AC) task. In this study, we investigated whether this improvement resulted from the difference in methods between the rotating tray task and previous tests, or whether it was the result of greater overall experience with delay of gratification tasks. Experiment 1 produced similar performance levels by capuchins monkeys in the RT and AC tasks when identical reward and temporal parameters were used. Experiment 2 demonstrated a similar result using reward amounts that were more similar to previous AC experiments with these monkeys. In Experiment 3, monkeys performed multiple versions of the AC task with varied reward and temporal parameters. Their self-control behavior was found to be dependent on the overall delay to reward consumption, rather than the overall reward amount ultimately consumed. These findings indicate that these capuchin monkeys’ self-control capacities were more likely to have improved across studies because of the greater experience they had with delay of gratification tasks. Experiment 4 and Experiment 5 tested new, task-naïve monkeys on both tasks, finding more limited evidence of self-control, and no evidence that one task was more beneficial than the other in promoting self-control. The results of this study suggest that future testing of this kind should focus on temporal parameters and reward magnitude parameters to establish accurate measures of delay of gratification capacity and development in this species and perhaps others. PMID:27298233
-
Circadian timed episodic-like memory - a bee knows what to do when, and also where.
Pahl, Mario; Zhu, Hong; Pix, Waltraud; Tautz, Juergen; Zhang, Shaowu
2007-10-01
This study investigates how the colour, shape and location of patterns could be memorized within a time frame. Bees were trained to visit two Y-mazes, one of which presented yellow vertical (rewarded) versus horizontal (non-rewarded) gratings at one site in the morning, while another presented blue horizontal (rewarded) versus vertical (non-rewarded) gratings at another site in the afternoon. The bees could perform well in the learning tests and various transfer tests, in which (i) all contextual cues from the learning test were present; (ii) the colour cues of the visual patterns were removed, but the location cue, the orientation of the visual patterns and the temporal cue still existed; (iii) the location cue was removed, but other contextual cues, i.e. the colour and orientation of the visual patterns and the temporal cue still existed; (iv) the location cue and the orientation cue of the visual patterns were removed, but the colour cue and temporal cue still existed; (v) the location cue, and the colour cue of the visual patterns were removed, but the orientation cue and the temporal cue still existed. The results reveal that the honeybee can recall the memory of the correct visual patterns by using spatial and/or temporal information. The relative importance of different contextual cues is compared and discussed. The bees' ability to integrate elements of circadian time, place and visual stimuli is akin to episodic-like memory; we have therefore named this kind of memory circadian timed episodic-like memory.
-
Scheres, Anouk; Dijkstra, Marianne; Ainslie, Eleanor; Balkan, Jaclyn; Reynolds, Brady; Sonuga-Barke, Edmund; Castellanos, F Xavier
2006-01-01
This study investigated whether age and ADHD symptoms affected choice preferences in children and adolescents when they chose between (1) small immediate rewards and larger delayed rewards and (2) small certain rewards and larger probabilistic uncertain rewards. A temporal discounting (TD) task and a probabilistic discounting (PD) task were used to measure the degree to which the subjective value of a large reward decreased as one had to wait longer for it (TD), and as the probability of obtaining it decreased (PD). Rewards used were small amounts of money. In the TD task, the large reward (10 cents) was delayed by between 0 and 30s, and the immediate reward varied in magnitude (0-10 cents). In the PD task, receipt of the large reward (10 cents) varied in likelihood, with probabilities of 0, 0.25, 0.5, 0.75, and 1.0 used, and the certain reward varied in magnitude (0-10 cents). Age and diagnostic group did not affect the degree of PD of rewards: All participants made choices so that total gains were maximized. As predicted, young children, aged 6-11 years (n = 25) demonstrated steeper TD of rewards than adolescents, aged 12-17 years (n = 21). This effect remained significant even when choosing the immediate reward did not shorten overall task duration. This, together with the lack of interaction between TD task version and age, suggests that steeper discounting in young children is driven by reward immediacy and not by delay aversion. Contrary to our predictions, participants with ADHD (n = 22) did not demonstrate steeper TD of rewards than controls (n = 24). These results raise the possibility that strong preferences for small immediate rewards in ADHD, as found in previous research, depend on factors such as total maximum gain and the use of fixed versus varied delay durations. The decrease in TD as observed in adolescents compared to children may be related to developmental changes in the (dorsolateral) prefrontal cortex. Future research needs to investigate these possibilities.
-
Sex differences in brain activation elicited by humor.
Azim, Eiman; Mobbs, Dean; Jo, Booil; Menon, Vinod; Reiss, Allan L
2005-11-08
With recent investigation beginning to reveal the cortical and subcortical neuroanatomical correlates of humor appreciation, the present event-related functional MRI (fMRI) study was designed to elucidate sex-specific recruitment of these humor related networks. Twenty healthy subjects (10 females) underwent fMRI scanning while subjectively rating 70 verbal and nonverbal achromatic cartoons as funny or unfunny. Data were analyzed by comparing blood oxygenation-level-dependent signal activation during funny and unfunny stimuli. Males and females share an extensive humor-response strategy as indicated by recruitment of similar brain regions: both activate the temporal-occipital junction and temporal pole, structures implicated in semantic knowledge and juxtaposition, and the inferior frontal gyrus, likely to be involved in language processing. Females, however, activate the left prefrontal cortex more than males, suggesting a greater degree of executive processing and language-based decoding. Females also exhibit greater activation of mesolimbic regions, including the nucleus accumbens, implying greater reward network response and possibly less reward expectation. These results indicate sex-specific differences in neural response to humor with implications for sex-based disparities in the integration of cognition and emotion.
-
An Imperfect Dopaminergic Error Signal Can Drive Temporal-Difference Learning
Potjans, Wiebke; Diesmann, Markus; Morrison, Abigail
2011-01-01
An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD) learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards. PMID:21589888
-
Response effort discounts the subjective value of rewards.
Nishiyama, Ryoji
2014-09-01
Factors associated with obtaining a reward, such as a temporal delay in receiving the reward, can influence the subjective value of the reward. Cognitive as well as physical response effort is also known to influence choice behaviors. The present study used hypothetical situations to assess whether response effort affects the subjective value of rewards. The results demonstrated that increasing response effort increases the amount of money that participants are willing to forgo to avoid engaging in work. An exponential as well as hyperbolic function provided a good fit for such discounting. The findings suggest that response effort discounts the subjective value of a reward as a function of its amount. Copyright © 2014 Elsevier B.V. All rights reserved.
-
Sleep deprivation alters effort discounting but not delay discounting of monetary rewards.
Libedinsky, Camilo; Massar, Stijn A A; Ling, Aiqing; Chee, Weiyan; Huettel, Scott A; Chee, Michael W L
2013-06-01
To determine whether sleep deprivation would affect the discounting of delayed rewards, of rewards entailing the expense of effort, or both. We measured rates of two types of reward discounting under conditions of rested wakefulness (RW) and sleep deprivation (SD). Delay discounting was defined as the willingness to accept smaller monetary rewards sooner rather than larger monetary rewards later. Effort discounting was defined as the willingness to accept smaller rewards that require less effort to obtain (e.g., typing a small number of letter strings backward) over larger but more effortful rewards (e.g., typing more letter strings to receive the reward). The first two experiments used a crossover design in which one session was conducted after a normal night of sleep (RW), and the other after a night without sleep (SD). The first experiment evaluated only temporal discounting whereas the second evaluated temporal and effort discounting. In the second experiment, the discounting tasks were repeatedly administered prior to the state comparisons to minimize the effects of order and/or repeated testing. In a third experiment, participants were studied only once in a between-subject evaluation of discounting across states. The study took place in a research laboratory. Seventy-seven healthy young adult participants: 20 in Experiment 1, 27 in Experiment 2, and 30 in Experiment 3. N/A. Sleep deprivation elicited increased effort discounting but did not affect delay discounting. The dissociable effects of sleep deprivation on two forms of discounting behavior suggest that they may have differing underlying neural mechanisms.
-
Olivo, Diana; Caba, Mario; Gonzalez-Lima, Francisco; Rodríguez-Landa, Juan F; Corona-Morales, Aleph A
2017-01-01
When food is restricted to a brief fixed period every day, animals show an increase in temperature, corticosterone concentration and locomotor activity for 2-3h before feeding time, termed food anticipatory activity. Mechanisms and neuroanatomical circuits responsible for food anticipatory activity remain unclear, and may involve both oscillators and networks related to temporal conditioning. Rabbit pups are nursed once-a-day so they represent a natural model of circadian food anticipatory activity. Food anticipatory behavior in pups may be associated with neural circuits that temporally anticipate feeding, while the nursing event may produce consummatory effects. Therefore, we used New Zealand white rabbit pups entrained to circadian feeding to investigate the hypothesis that structures related to reward expectation and conditioned emotional responses would show a metabolic rhythm anticipatory of the nursing event, different from that shown by structures related to reward delivery. Quantitative cytochrome oxidase histochemistry was used to measure regional brain metabolic activity at eight different times during the day. We found that neural metabolism peaked before nursing, during food anticipatory behavior, in nuclei of the extended amygdala (basolateral, medial and central nuclei, bed nucleus of the stria terminalis), lateral septum and accumbens core. After pups were fed, however, maximal metabolic activity was expressed in the accumbens shell, caudate, putamen and cortical amygdala. Neural and behavioral activation persisted when animals were fasted by two cycles, at the time of expected nursing. These findings suggest that metabolic activation of amygdala-septal-accumbens circuits involved in temporal conditioning may contribute to food anticipatory activity. Copyright © 2016 Elsevier B.V. All rights reserved.
-
Self-control assessments of capuchin monkeys with the rotating tray task and the accumulation task.
Beran, Michael J; Perdue, Bonnie M; Rossettie, Mattea S; James, Brielle T; Whitham, Will; Walker, Bradlyn; Futch, Sara E; Parrish, Audrey E
2016-08-01
Recent studies of delay of gratification in capuchin monkeys using a rotating tray (RT) task have shown improved self-control performance in these animals in comparison to the accumulation (AC) task. In this study, we investigated whether this improvement resulted from the difference in methods between the rotating tray task and previous tests, or whether it was the result of greater overall experience with delay of gratification tasks. Experiment 1 produced similar performance levels by capuchins monkeys in the RT and AC tasks when identical reward and temporal parameters were used. Experiment 2 demonstrated a similar result using reward amounts that were more similar to previous AC experiments with these monkeys. In Experiment 3, monkeys performed multiple versions of the AC task with varied reward and temporal parameters. Their self-control behavior was found to be dependent on the overall delay to reward consumption, rather than the overall reward amount ultimately consumed. These findings indicate that these capuchin monkeys' self-control capacities were more likely to have improved across studies because of the greater experience they had with delay of gratification tasks. Experiment 4 and Experiment 5 tested new, task-naïve monkeys on both tasks, finding more limited evidence of self-control, and no evidence that one task was more beneficial than the other in promoting self-control. The results of this study suggest that future testing of this kind should focus on temporal parameters and reward magnitude parameters to establish accurate measures of delay of gratification capacity and development in this species and perhaps others. Copyright © 2016 Elsevier B.V. All rights reserved.
-
Phasic dopamine signals: from subjective reward value to formal economic utility
Schultz, Wolfram; Carelli, Regina M; Wightman, R Mark
2015-01-01
Although rewards are physical stimuli and objects, their value for survival and reproduction is subjective. The phasic, neurophysiological and voltammetric dopamine reward prediction error response signals subjective reward value. The signal incorporates crucial reward aspects such as amount, probability, type, risk, delay and effort. Differences of dopamine release dynamics with temporal delay and effort in rodents may derive from methodological issues and require further study. Recent designs using concepts and behavioral tools from experimental economics allow to formally characterize the subjective value signal as economic utility and thus to establish a neuronal value function. With these properties, the dopamine response constitutes a utility prediction error signal. PMID:26719853
-
Temporal discounting rates and their relation to exercise behavior in older adults.
Tate, Linda M; Tsai, Pao-Feng; Landes, Reid D; Rettiganti, Mallikarjuna; Lefler, Leanne L
2015-12-01
As our nation's population ages, the rates of chronic illness and disability are expected to increase significantly. Despite the knowledge that exercise may prevent chronic disease and promote health among older adults, many still are inactive. Factors related to exercise behaviors have been explored in recent years. However, temporal discounting is a motivational concept that has not been explored in regard to exercise in older adults. Temporal discounting is a decision making process by which an individual chooses a smaller more immediate reward over a larger delayed reward. The aim of this study was to determine if temporal discounting rates vary between exercising and non-exercising older adults. This study used cross-sectional survey of 137 older adults living in the community. Older adults were recruited from 11 rural Arkansas churches. The Kirby delay-discounting Monetary Choice Questionnaire was used to collect discounting rates and then bivariate analysis was performed to compare temporal discounting rate between the exercisers and non-exercisers. Finally, multivariate analysis was used to compare discounting rate controlling for other covariates. The results indicated that exercising older adults display lower temporal discounting rates than non-exercising older adults. After controlling for education, exercisers still have lower temporal discounting rates than non-exercisers (p<0.001). These findings are important as several chronic health conditions relate to lack of exercise especially in older adults. This research suggests that if we can find appropriate incentives for discounting individuals, some type of immediate reward, then potentially we can design programs to engage and retain older adults in exercise. Copyright © 2015 Elsevier Inc. All rights reserved.
-
Premkumar, Preethi; Fannon, Dominic; Kuipers, Elizabeth; Simmons, Andrew; Frangou, Sophia; Kumari, Veena
2008-01-01
Cognitive decision-making is known to be deficient, but relatively less is known about emotional decision-making in schizophrenia. The Iowa gambling task (IGT) is considered a reliable probe of emotional decision-making and believed to reflect orbitofrontal cortex (OFC) function. The expectancy-valence model of IGT performance implicates three dissociable components, namely, attention to reward, memory for past, relative to recent, outcomes and impulsivity in emotional decision-making. We examined IGT performance, its three components, and their grey matter volume (GMV) correlates in 75 stable patients with schizophrenia, relative to 25 healthy individuals. Patients, relative to controls, showed impaired IGT performance and poor memory for past, relative to recent, outcomes. IGT performance correlated with GMV in the OFC in controls, but not patients. There were associations between (a) attention to reward and GMV in the frontal, temporal, parietal and striatal regions in controls, and in the temporal and thalamic regions in patients, (b) memory for past outcomes and GMV in the temporal region in controls, and the frontal and temporal regions in patients, and (c) low impulsivity and greater GMV in the frontal, temporal, posterior cingulate and occipital regions in controls, and in the frontal, temporal and posterior cingulate regions in patients. Most IGT-GMV associations were stronger in controls. It is concluded that (i) poor memory, rather than less attention to reward or impulsivity, contributes to IGT performance deficit, and (ii) the relationship of IGT performance and its components with GMVs especially in the frontal and temporal lobes is lost or attenuated in schizophrenia.
-
A general theory of intertemporal decision-making and the perception of time.
Namboodiri, Vijay M K; Mihalas, Stefan; Marton, Tanya M; Hussain Shuler, Marshall G
2014-01-01
Animals and humans make decisions based on their expected outcomes. Since relevant outcomes are often delayed, perceiving delays and choosing between earlier vs. later rewards (intertemporal decision-making) is an essential component of animal behavior. The myriad observations made in experiments studying intertemporal decision-making and time perception have not yet been rationalized within a single theory. Here we present a theory-Training-Integrated Maximized Estimation of Reinforcement Rate (TIMERR)-that explains a wide variety of behavioral observations made in intertemporal decision-making and the perception of time. Our theory postulates that animals make intertemporal choices to optimize expected reward rates over a limited temporal window which includes a past integration interval-over which experienced reward rate is estimated-as well as the expected delay to future reward. Using this theory, we derive mathematical expressions for both the subjective value of a delayed reward and the subjective representation of the delay. A unique contribution of our work is in finding that the past integration interval directly determines the steepness of temporal discounting and the non-linearity of time perception. In so doing, our theory provides a single framework to understand both intertemporal decision-making and time perception.
-
A general theory of intertemporal decision-making and the perception of time
Namboodiri, Vijay M. K.; Mihalas, Stefan; Marton, Tanya M.; Hussain Shuler, Marshall G.
2014-01-01
Animals and humans make decisions based on their expected outcomes. Since relevant outcomes are often delayed, perceiving delays and choosing between earlier vs. later rewards (intertemporal decision-making) is an essential component of animal behavior. The myriad observations made in experiments studying intertemporal decision-making and time perception have not yet been rationalized within a single theory. Here we present a theory—Training-Integrated Maximized Estimation of Reinforcement Rate (TIMERR)—that explains a wide variety of behavioral observations made in intertemporal decision-making and the perception of time. Our theory postulates that animals make intertemporal choices to optimize expected reward rates over a limited temporal window which includes a past integration interval—over which experienced reward rate is estimated—as well as the expected delay to future reward. Using this theory, we derive mathematical expressions for both the subjective value of a delayed reward and the subjective representation of the delay. A unique contribution of our work is in finding that the past integration interval directly determines the steepness of temporal discounting and the non-linearity of time perception. In so doing, our theory provides a single framework to understand both intertemporal decision-making and time perception. PMID:24616677
-
Working-memory load and temporal myopia in dynamic decision making.
Worthy, Darrell A; Otto, A Ross; Maddox, W Todd
2012-11-01
We examined the role of working memory (WM) in dynamic decision making by having participants perform decision-making tasks under single-task or dual-task conditions. In 2 experiments participants performed dynamic decision-making tasks in which they chose 1 of 2 options on each trial. The decreasing option always gave a larger immediate reward but caused future rewards for both options to decrease. The increasing option always gave a smaller immediate reward but caused future rewards for both options to increase. In each experiment we manipulated the reward structure such that the decreasing option was the optimal choice in 1 condition and the increasing option was the optimal choice in the other condition. Behavioral results indicated that dual-task participants selected the immediately rewarding decreasing option more often, and single-task participants selected the increasing option more often, regardless of which option was optimal. Thus, dual-task participants performed worse on 1 type of task but better on the other type. Modeling results showed that single-task participants' data were most often best fit by a win-stay, lose-shift (WSLS) rule-based model that tracked differences across trials, and dual-task participants' data were most often best fit by a Softmax reinforcement learning model that tracked recency-weighted average rewards for each option. This suggests that manipulating WM load affects the degree to which participants focus on the immediate versus delayed consequences of their actions and whether they employ a rule-based WSLS strategy, but it does not necessarily affect how well people weigh the immediate versus delayed benefits when determining the long-term utility of each option.
-
Monetary reward speeds up voluntary saccades.
Chen, Lewis L; Chen, Y Mark; Zhou, Wu; Mustain, William D
2014-01-01
Past studies have shown that reward contingency is critical for sensorimotor learning, and reward expectation speeds up saccades in animals. Whether monetary reward speeds up saccades in human remains unknown. Here we addressed this issue by employing a conditional saccade task, in which human subjects performed a series of non-reflexive, visually-guided horizontal saccades. The subjects were (or were not) financially compensated for making a saccade in response to a centrally-displayed visual congruent (or incongruent) stimulus. Reward modulation of saccadic velocities was quantified independently of the amplitude-velocity coupling. We found that reward expectation significantly sped up voluntary saccades up to 30°/s, and the reward modulation was consistent across tests. These findings suggest that monetary reward speeds up saccades in human in a fashion analogous to how juice reward sped up saccades in monkeys. We further noticed that the idiosyncratic nasal-temporal velocity asymmetry was highly consistent regardless of test order, and its magnitude was not correlated with the magnitude of reward modulation. This suggests that reward modulation and the intrinsic velocity asymmetry may be governed by separate mechanisms that regulate saccade generation.
-
Trait Anticipatory Pleasure Predicts Effort Expenditure for Reward
Geaney, Joachim T.; Treadway, Michael T.; Smillie, Luke D.
2015-01-01
Research in motivation and emotion has been increasingly influenced by the perspective that processes underpinning the motivated approach of rewarding goals are distinct from those underpinning enjoyment during reward consummation. This distinction recently inspired the construction of the Temporal Experience of Pleasure Scale (TEPS), a self-report measure that distinguishes trait anticipatory pleasure (pre-reward feelings of desire) from consummatory pleasure (feelings of enjoyment and gratification upon reward attainment). In a university community sample (N = 97), we examined the TEPS subscales as predictors of (1) the willingness to expend effort for monetary rewards, and (2) affective responses to a pleasant mood induction procedure. Results showed that both anticipatory pleasure and a well-known trait measure of reward motivation predicted effort-expenditure for rewards when the probability of being rewarded was relatively low. Against expectations, consummatory pleasure was unrelated to induced pleasant affect. Taken together, our findings provide support for the validity of the TEPS anticipatory pleasure scale, but not the consummatory pleasure scale. PMID:26115223
-
Palombo, Daniela J.; Keane, Margaret M.; Verfaellie, Mieke
2014-01-01
In the present study, we investigated the effect of medial temporal lobe (MTL) damage on human decision making in the context of reward-based intertemporal choice. During intertemporal choice, humans typically devalue (or discount) a future reward to account for its delayed arrival (e.g., preferring $30 now over $42 in 2 months), but this effect is attenuated when participants engage in episodic future thinking, i.e., project themselves into the future to imagine a specific event. We hypothesized that this attenuation would be selectively impaired in amnesic patients, who have deficits in episodic future thinking. Replicating previous work, in a standard intertemporal choice task, amnesic patients showed temporal discounting indices similar to healthy controls. Consistent with our hypothesis, while healthy controls demonstrated attenuated temporal discounting in a condition that required participants first to engage in episodic future thinking (e.g., to imagine spending $42 at a theatre in 2 months), amnesic patients failed to demonstrate this effect. Moreover, as expected, amnesic patients’ narratives were less episodically rich than those of controls. These findings extend the range of tasks that are shown to be MTL dependent to include not only memory-based decision-making tasks but also future-oriented ones. PMID:25284804
-
ADHD and Delay Aversion: The Influence of Non-Temporal Stimulation on Choice for Delayed Rewards
ERIC Educational Resources Information Center
Antrop, Inge; Stock, Pieter; Verte, Sylvie; Wiersema, Jan Roelt; Baeyens, Dieter; Roeyers, Herbert
2006-01-01
Background: Delay aversion, the motivation to escape or avoid delay, results in preference for small immediate over large delayed rewards. Delay aversion has been proposed as one distinctive psychological process that may underlie the behavioural symptoms and cognitive deficits of attention deficit/hyperactivity disorder (ADHD). Furthermore, the…
-
Mathiak, Krystyna A; Klasen, Martin; Weber, René; Ackermann, Hermann; Shergill, Sukhwinder S; Mathiak, Klaus
2011-07-12
Violent content in video games evokes many concerns but there is little research concerning its rewarding aspects. It was demonstrated that playing a video game leads to striatal dopamine release. It is unclear, however, which aspects of the game cause this reward system activation and if violent content contributes to it. We combined functional Magnetic Resonance Imaging (fMRI) with individual affect measures to address the neuronal correlates of violence in a video game. Thirteen male German volunteers played a first-person shooter game (Tactical Ops: Assault on Terror) during fMRI measurement. We defined success as eliminating opponents, and failure as being eliminated themselves. Affect was measured directly before and after game play using the Positive and Negative Affect Schedule (PANAS). Failure and success events evoked increased activity in visual cortex but only failure decreased activity in orbitofrontal cortex and caudate nucleus. A negative correlation between negative affect and responses to failure was evident in the right temporal pole (rTP). The deactivation of the caudate nucleus during failure is in accordance with its role in reward-prediction error: it occurred whenever subject missed an expected reward (being eliminated rather than eliminating the opponent). We found no indication that violence events were directly rewarding for the players. We addressed subjective evaluations of affect change due to gameplay to study the reward system. Subjects reporting greater negative affect after playing the game had less rTP activity associated with failure. The rTP may therefore be involved in evaluating the failure events in a social context, to regulate the players' mood.
-
Huertas, Marco A; Schwettmann, Sarah E; Shouval, Harel Z
2016-01-01
The ability to maximize reward and avoid punishment is essential for animal survival. Reinforcement learning (RL) refers to the algorithms used by biological or artificial systems to learn how to maximize reward or avoid negative outcomes based on past experiences. While RL is also important in machine learning, the types of mechanistic constraints encountered by biological machinery might be different than those for artificial systems. Two major problems encountered by RL are how to relate a stimulus with a reinforcing signal that is delayed in time (temporal credit assignment), and how to stop learning once the target behaviors are attained (stopping rule). To address the first problem synaptic eligibility traces were introduced, bridging the temporal gap between a stimulus and its reward. Although, these were mere theoretical constructs, recent experiments have provided evidence of their existence. These experiments also reveal that the presence of specific neuromodulators converts the traces into changes in synaptic efficacy. A mechanistic implementation of the stopping rule usually assumes the inhibition of the reward nucleus; however, recent experimental results have shown that learning terminates at the appropriate network state even in setups where the reward nucleus cannot be inhibited. In an effort to describe a learning rule that solves the temporal credit assignment problem and implements a biologically plausible stopping rule, we proposed a model based on two separate synaptic eligibility traces, one for long-term potentiation (LTP) and one for long-term depression (LTD), each obeying different dynamics and having different effective magnitudes. The model has been shown to successfully generate stable learning in recurrent networks. Although, the model assumes the presence of a single neuromodulator, evidence indicates that there are different neuromodulators for expressing the different traces. What could be the role of different neuromodulators for expressing the LTP and LTD traces? Here we expand on our previous model to include several neuromodulators, and illustrate through various examples how different these contribute to learning reward-timing within a wide set of training paradigms and propose further roles that multiple neuromodulators can play in encoding additional information of the rewarding signal.
-
Immaturities in Reward Processing and Its Influence on Inhibitory Control in Adolescence
Terwilliger, R.; Teslovich, T.; Velanova, K.; Luna, B.
2010-01-01
The nature of immature reward processing and the influence of rewards on basic elements of cognitive control during adolescence are currently not well understood. Here, during functional magnetic resonance imaging, healthy adolescents and adults performed a modified antisaccade task in which trial-by-trial reward contingencies were manipulated. The use of a novel fast, event-related design enabled developmental differences in brain function underlying temporally distinct stages of reward processing and response inhibition to be assessed. Reward trials compared with neutral trials resulted in faster correct inhibitory responses across ages and in fewer inhibitory errors in adolescents. During reward trials, the blood oxygen level–dependent signal was attenuated in the ventral striatum in adolescents during cue assessment, then overactive during response preparation, suggesting limitations during adolescence in reward assessment and heightened reactivity in anticipation of reward compared with adults. Importantly, heightened activity in the frontal cortex along the precentral sulcus was also observed in adolescents during reward-trial response preparation, suggesting reward modulation of oculomotor control regions supporting correct inhibitory responding. Collectively, this work characterizes specific immaturities in adolescent brain systems that support reward processing and describes the influence of reward on inhibitory control. In sum, our findings suggest mechanisms that may underlie adolescents’ vulnerability to poor decision-making and risk-taking behavior. PMID:19875675
-
ERIC Educational Resources Information Center
Demurie, Ellen; Roeyers, Herbert; Baeyens, Dieter; Sonuga-Barke, Edmund
2012-01-01
It has been difficult to differentiate attention-deficit/hyperactivity disorder (ADHD) and autism spectrum disorder (ASD) in terms of some aspects of their cognitive profile. While both show deficits in executive functions, it has been suggested that they may differ in their response to monetary reward. For instance, children with ADHD prefer…
-
Decision-making in the adolescent brain.
Blakemore, Sarah-Jayne; Robbins, Trevor W
2012-09-01
Adolescence is characterized by making risky decisions. Early lesion and neuroimaging studies in adults pointed to the ventromedial prefrontal cortex and related structures as having a key role in decision-making. More recent studies have fractionated decision-making processes into its various components, including the representation of value, response selection (including inter-temporal choice and cognitive control), associative learning, and affective and social aspects. These different aspects of decision-making have been the focus of investigation in recent studies of the adolescent brain. Evidence points to a dissociation between the relatively slow, linear development of impulse control and response inhibition during adolescence versus the nonlinear development of the reward system, which is often hyper-responsive to rewards in adolescence. This suggests that decision-making in adolescence may be particularly modulated by emotion and social factors, for example, when adolescents are with peers or in other affective ('hot') contexts.
-
The Iowa Gambling Task and the three fallacies of dopamine in gambling disorder
Linnet, Jakob
2013-01-01
Gambling disorder sufferers prefer immediately larger rewards despite long term losses on the Iowa Gambling Task (IGT), and these impairments are associated with dopamine dysfunctions. Dopamine is a neurotransmitter linked with temporal and structural dysfunctions in substance use disorder, which has supported the idea of impaired decision-making and dopamine dysfunctions in gambling disorder. However, evidence from substance use disorders cannot be directly transferred to gambling disorder. This article focuses on three hypotheses of dopamine dysfunctions in gambling disorder, which appear to be “fallacies,” i.e., have not been supported in a series of positron emission tomography (PET) studies. The first “fallacy” suggests that gambling disorder sufferers have lower dopamine receptor availability, as seen in substance use disorders. However, no evidence supported this hypothesis. The second “fallacy” suggests that maladaptive decision-making in gambling disorder is associated with higher dopamine release during gambling. No evidence supported the hypothesis, and the literature on substance use disorders offers limited support for this hypothesis. The third “fallacy” suggests that maladaptive decision-making in gambling disorder is associated with higher dopamine release during winning. The evidence did not support this hypothesis either. Instead, dopaminergic coding of reward prediction and uncertainty might better account for dopamine dysfunctions in gambling disorder. Studies of reward prediction and reward uncertainty show a sustained dopamine response toward stimuli with maximum uncertainty, which may explain the continued dopamine release and gambling despite losses in gambling disorder. The findings from the studies presented here are consistent with the notion of dopaminergic dysfunctions of reward prediction and reward uncertainty signals in gambling disorder. PMID:24115941
-
A Drosophila model for alcohol reward
Kaun, K.R.; Azanchi, R.; Maung, Z.; Hirsh, J.; Heberlein, U.
2014-01-01
The rewarding properties of drugs contribute to the development of abuse and addiction. Here we present a new assay to investigate the motivational properties of ethanol in the genetically tractable model, Drosophila melanogaster. Flies learn to associate cues with ethanol intoxication and, although transiently aversive, the experience leads to a long-lasting attraction for the ethanol-paired cue, implying that intoxication is rewarding. Temporally blocking transmission in dopaminergic neurons revealed that flies require activation of these neurons to express, but not develop, conditioned preference for ethanol-associated cues. Moreover, flies acquire, consolidate, and retrieve these rewarding memories using distinct sets of neurons of the mushroom body. Finally, mutations in scabrous, encoding a fibrinogen-related peptide that regulates Notch signaling, disrupt the formation of memories for ethanol reward. Our results thus establish that Drosophila can be useful in understanding the molecular, genetic and neural mechanisms underling the rewarding properties of ethanol. PMID:21499254
-
Overlapping neural systems represent cognitive effort and reward anticipation.
Vassena, Eliana; Silvetti, Massimo; Boehler, Carsten N; Achten, Eric; Fias, Wim; Verguts, Tom
2014-01-01
Anticipating a potential benefit and how difficult it will be to obtain it are valuable skills in a constantly changing environment. In the human brain, the anticipation of reward is encoded by the Anterior Cingulate Cortex (ACC) and Striatum. Naturally, potential rewards have an incentive quality, resulting in a motivational effect improving performance. Recently it has been proposed that an upcoming task requiring effort induces a similar anticipation mechanism as reward, relying on the same cortico-limbic network. However, this overlapping anticipatory activity for reward and effort has only been investigated in a perceptual task. Whether this generalizes to high-level cognitive tasks remains to be investigated. To this end, an fMRI experiment was designed to investigate anticipation of reward and effort in cognitive tasks. A mental arithmetic task was implemented, manipulating effort (difficulty), reward, and delay in reward delivery to control for temporal confounds. The goal was to test for the motivational effect induced by the expectation of bigger reward and higher effort. The results showed that the activation elicited by an upcoming difficult task overlapped with higher reward prospect in the ACC and in the striatum, thus highlighting a pivotal role of this circuit in sustaining motivated behavior.
-
2011-01-01
Background Violent content in video games evokes many concerns but there is little research concerning its rewarding aspects. It was demonstrated that playing a video game leads to striatal dopamine release. It is unclear, however, which aspects of the game cause this reward system activation and if violent content contributes to it. We combined functional Magnetic Resonance Imaging (fMRI) with individual affect measures to address the neuronal correlates of violence in a video game. Results Thirteen male German volunteers played a first-person shooter game (Tactical Ops: Assault on Terror) during fMRI measurement. We defined success as eliminating opponents, and failure as being eliminated themselves. Affect was measured directly before and after game play using the Positive and Negative Affect Schedule (PANAS). Failure and success events evoked increased activity in visual cortex but only failure decreased activity in orbitofrontal cortex and caudate nucleus. A negative correlation between negative affect and responses to failure was evident in the right temporal pole (rTP). Conclusions The deactivation of the caudate nucleus during failure is in accordance with its role in reward-prediction error: it occurred whenever subject missed an expected reward (being eliminated rather than eliminating the opponent). We found no indication that violence events were directly rewarding for the players. We addressed subjective evaluations of affect change due to gameplay to study the reward system. Subjects reporting greater negative affect after playing the game had less rTP activity associated with failure. The rTP may therefore be involved in evaluating the failure events in a social context, to regulate the players' mood. PMID:21749711
-
Schultz, Wolfram
2004-04-01
Neurons in a small number of brain structures detect rewards and reward-predicting stimuli and are active during the expectation of predictable food and liquid rewards. These neurons code the reward information according to basic terms of various behavioural theories that seek to explain reward-directed learning, approach behaviour and decision-making. The involved brain structures include groups of dopamine neurons, the striatum including the nucleus accumbens, the orbitofrontal cortex and the amygdala. The reward information is fed to brain structures involved in decision-making and organisation of behaviour, such as the dorsolateral prefrontal cortex and possibly the parietal cortex. The neural coding of basic reward terms derived from formal theories puts the neurophysiological investigation of reward mechanisms on firm conceptual grounds and provides neural correlates for the function of rewards in learning, approach behaviour and decision-making.
-
Motivated To Win: Relationship between Anticipatory and Outcome Reward-Related Neural Activity
Nusslock, Robin
2015-01-01
Reward-processing involves two temporal stages characterized by two distinct neural processes: reward-anticipation and reward-outcome. Intriguingly, very little research has examined the relationship between neural processes involved in reward-anticipation and reward-outcome. To investigate this, one needs to consider the heterogeneity of reward-processing within each stage. To identify different stages of reward processing, we adapted a reward time-estimation task. While EEG data were recorded, participants were instructed to button-press 3.5 s after the onset of an Anticipation-Cue and received monetary reward for good time-estimation on the Reward trials, but not on No-Reward trials. We first separated reward-anticipation into event related potentials (ERPs) occurring at three sub-stages: reward/no-reward cue-evaluation, motor-preparation and feedback-anticipation. During reward/no-reward cue-evaluation, the Reward-Anticipation Cue led to a smaller N2 and larger P3. During motor-preparation, we report, for the first time, that the Reward-Anticipation Cue enhanced the Readiness Potential (RP), starting approximately 1 s before movement. At the subsequent feedback-anticipation stage, the Reward-Anticipation Cue elevated the Stimulus-Preceding Negativity (SPN). We also separated reward-outcome ERPs into different components occurring at different time-windows: the Feedback-Related Negativity (FRN), Feedback-P3 (FB-P3) and Late-Positive Potentials (LPP). Lastly, we examined the relationship between reward-anticipation and reward-outcome ERPs. We report that individual-differences in specific reward-anticipation ERPs uniquely predicted specific reward-outcome ERPs. In particular, the reward-anticipation Early-RP (1 to .8 s before movement) predicted early reward-outcome ERPs (FRN and FB-P3), whereas, the reward-anticipation SPN most strongly predicted a later reward-outcome ERP (LPP). Results have important implications for understanding the nature of the relationship between reward-anticipation and reward-outcome neural-processes. PMID:26433773
-
Hsu, Chia-Fen; Sonuga-Barke, Edmund J S
2016-08-01
fMRI studies have implicated the medial prefrontal cortex and medial temporal lobe, components of the default mode network (DMN), in episodic prospection. This study compared quantitative EEG localized to these DMN regions during prospection and during resting and while waiting for rewards. EEG was recorded in twenty-two adults while they were asked to (i) envision future monetary episodes; (ii) wait for rewards and (iii) rest. Activation sources were localized to core DMN regions. EEG power and phase coherence were compared across conditions. Prospection, compared to resting and waiting, was associated with reduced power in the medial prefrontal gyrus and increased power in the bilateral medial temporal gyrus across frequency bands as well as greater phase synchrony between these regions in the delta band. The current quantitative EEG analysis confirms prior fMRI research suggesting that medial prefrontal and medial temporal gyrus interactions are central to the capacity for episodic prospection. Copyright © 2016 Elsevier B.V. All rights reserved.
-
Katz, Andrea C.; Sarapas, Casey; Bishop, Jeffrey R.; Patel, Shitalben R.; Shankman, Stewart A.
2014-01-01
The Val158Met rs4680 polymorphism in the COMT gene regulates dopamine catabolism in the prefrontal cortex (PFC). Dopamine’s involvement in reward experience suggests those with the Met variant may exhibit trait-level sensitivity to reward due to more post-synaptic dopamine in the PFC. A physiological mediator of this association may be greater relative left asymmetry in the PFC, a putative biomarker for trait positive emotionality. Electroencephalograms of 120 participants were measured during a task that assesses two aspects of reward processing: pre-reward anticipation and post-reward consummatory affect. Participants provided genetics samples and completed the Temporal Experience of Pleasure Scale, which assesses trait-level anticipatory and consummatory positive affect. Met carriers had higher TEPS-Consummatory scores. This effect was mediated by greater relative left activation in the post-reward phase of the task. No effects were observed for the pre-reward phase. Results suggest that frontal asymmetry is an endophenotype between COMT genotype and trait reward responsivity. PMID:25195915
-
Evidence for a shared representation of sequential cues that engage sign-tracking.
Smedley, Elizabeth B; Smith, Kyle S
2018-06-19
Sign-tracking is a phenomenon whereby cues that predict rewards come to acquire their own motivational value (incentive salience) and attract appetitive behavior. Typically, sign-tracking paradigms have used single auditory, visual, or lever cues presented prior to a reward delivery. Yet, real world examples of events often can be predicted by a sequence of cues. We have shown that animals will sign-track to multiple cues presented in temporal sequence, and with time develop a bias in responding toward a reward distal cue over a reward proximal cue. Further, extinction of responding to the reward proximal cue directly decreases responding to the reward distal cue. One possible explanation of this result is that serial cues become representationally linked with one another. Here we provide further support of this by showing that extinction of responding to a reward distal cue directly reduces responding to a reward proximal cue. We suggest that the incentive salience of one cue can influence the incentive salience of the other cue. Copyright © 2018. Published by Elsevier B.V.
-
Carton, J S
1996-01-01
Substantial research indicates that tangible rewards, such as money, prizes, and tokens, decrease response rates by undermining intrinsic motivation. In contrast, praise appears to increase response rates by enhancing intrinsic motivation. Based on their interpretation of available evidence, many social-cognitive researchers warn not to use tangible rewards in applied settings and to use praise instead. Furthermore, they suggest that the differential effects of the two types of rewards on intrinsic motivation cannot be explained using principles of operant psychology. Cognitive evaluation theory provides one of the most recent and widely cited social-cognitive explanations for the different effects of the two types of rewards on intrinsic motivation (Deci & Ryan, 1985). However, a review of existing research found little support for the explanations based on this theory and revealed three potential confounding effects: (a) temporal contiguity, (b) the number of reward administrations, and (c) discriminative stimuli associated with reward availability. These three confounding factors provide explanations for the effects of tangible rewards and praise on intrinsic motivation that are consistent with principles of operant psychology.
-
Carton, John S.
1996-01-01
Substantial research indicates that tangible rewards, such as money, prizes, and tokens, decrease response rates by undermining intrinsic motivation. In contrast, praise appears to increase response rates by enhancing intrinsic motivation. Based on their interpretation of available evidence, many social-cognitive researchers warn not to use tangible rewards in applied settings and to use praise instead. Furthermore, they suggest that the differential effects of the two types of rewards on intrinsic motivation cannot be explained using principles of operant psychology. Cognitive evaluation theory provides one of the most recent and widely cited social-cognitive explanations for the different effects of the two types of rewards on intrinsic motivation (Deci & Ryan, 1985). However, a review of existing research found little support for the explanations based on this theory and revealed three potential confounding effects: (a) temporal contiguity, (b) the number of reward administrations, and (c) discriminative stimuli associated with reward availability. These three confounding factors provide explanations for the effects of tangible rewards and praise on intrinsic motivation that are consistent with principles of operant psychology. PMID:22478261
-
Solway, A.; Botvinick, M.
2013-01-01
Recent work has given rise to the view that reward-based decision making is governed by two key controllers: a habit system, which stores stimulus-response associations shaped by past reward, and a goal-oriented system that selects actions based on their anticipated outcomes. The current literature provides a rich body of computational theory addressing habit formation, centering on temporal-difference learning mechanisms. Less progress has been made toward formalizing the processes involved in goal-directed decision making. We draw on recent work in cognitive neuroscience, animal conditioning, cognitive and developmental psychology and machine learning, to outline a new theory of goal-directed decision making. Our basic proposal is that the brain, within an identifiable network of cortical and subcortical structures, implements a probabilistic generative model of reward, and that goal-directed decision making is effected through Bayesian inversion of this model. We present a set of simulations implementing the account, which address benchmark behavioral and neuroscientific findings, and which give rise to a set of testable predictions. We also discuss the relationship between the proposed framework and other models of decision making, including recent models of perceptual choice, to which our theory bears a direct connection. PMID:22229491
-
Underconnectivity between voice-selective cortex and reward circuitry in children with autism.
Abrams, Daniel A; Lynch, Charles J; Cheng, Katherine M; Phillips, Jennifer; Supekar, Kaustubh; Ryali, Srikanth; Uddin, Lucina Q; Menon, Vinod
2013-07-16
Individuals with autism spectrum disorders (ASDs) often show insensitivity to the human voice, a deficit that is thought to play a key role in communication deficits in this population. The social motivation theory of ASD predicts that impaired function of reward and emotional systems impedes children with ASD from actively engaging with speech. Here we explore this theory by investigating distributed brain systems underlying human voice perception in children with ASD. Using resting-state functional MRI data acquired from 20 children with ASD and 19 age- and intelligence quotient-matched typically developing children, we examined intrinsic functional connectivity of voice-selective bilateral posterior superior temporal sulcus (pSTS). Children with ASD showed a striking pattern of underconnectivity between left-hemisphere pSTS and distributed nodes of the dopaminergic reward pathway, including bilateral ventral tegmental areas and nucleus accumbens, left-hemisphere insula, orbitofrontal cortex, and ventromedial prefrontal cortex. Children with ASD also showed underconnectivity between right-hemisphere pSTS, a region known for processing speech prosody, and the orbitofrontal cortex and amygdala, brain regions critical for emotion-related associative learning. The degree of underconnectivity between voice-selective cortex and reward pathways predicted symptom severity for communication deficits in children with ASD. Our results suggest that weak connectivity of voice-selective cortex and brain structures involved in reward and emotion may impair the ability of children with ASD to experience speech as a pleasurable stimulus, thereby impacting language and social skill development in this population. Our study provides support for the social motivation theory of ASD.
-
Locey, Matthew L.; Jones, Bryan A.; Rachlin, Howard
2012-01-01
Laboratory studies of choice and decision making among real monetary rewards typically use smaller real rewards than those common in real life. When laboratory rewards are large, they are almost always hypothetical. In applying laboratory results meaningfully to real-life situations, it is important to know the extent to which choices among hypothetical rewards correspond to choices among real rewards and whether variation of the magnitude of hypothetical rewards affects behavior in meaningful ways. The present study compared real and hypothetical monetary rewards in two experiments. In Experiment 1, participants played a temporal discounting game that incorporates the logic of a repeated prisoner’s-dilemma (PD) type game versus tit-for-tat; choice of one alternative (“defection” in PD terminology) resulted in a small-immediate reward; choice of the other alternative (“cooperation” in PD terminology) resulted in a larger reward delayed until the following trial. The larger-delayed reward was greater for half of the groups than for the other half. Rewards also differed in type across groups: multiples of real nickels, hypothetical nickels or hypothetical hundred-dollar bills. All groups significantly increased choice of the larger delayed reward over the 40 trials of the experiment. Over the last 10 trials, cooperation was significantly higher when the difference between larger and smaller hypothetical rewards was greater. Reward type (real or hypothetical) made no significant difference in cooperation. In Experiment 2, real and hypothetical rewards were compared in social discounting – the decrease in value to the giver of a reward as social distance increases to the receiver of the reward. Social discount rates were well described by a hyperbolic function. Discounting rates for real and hypothetical rewards did not significantly differ. These results add to the evidence that results of experiments with hypothetical rewards validly apply in everyday life. PMID:22582110
-
Pavlovian reward learning underlies value driven attentional capture.
Bucker, Berno; Theeuwes, Jan
2017-02-01
Recent evidence shows that distractors that signal high compared to low reward availability elicit stronger attentional capture, even when this is detrimental for task-performance. This suggests that simply correlating stimuli with reward administration, rather than their instrumental relationship with obtaining reward, produces value-driven attentional capture. However, in previous studies, reward delivery was never response independent, as only correct responses were rewarded, nor was it completely task-irrelevant, as the distractor signaled the magnitude of reward that could be earned on that trial. In two experiments, we ensured that associative reward learning was completely response independent by letting participants perform a task at fixation, while high and low rewards were automatically administered following the presentation of task-irrelevant colored stimuli in the periphery (Experiment 1) or at fixation (Experiment 2). In a following non-reward test phase, using the additional singleton paradigm, the previously reward signaling stimuli were presented as distractors to assess truly task-irrelevant value driven attentional capture. The results showed that high compared to low reward-value associated distractors impaired performance, and thus captured attention more strongly. This suggests that genuine Pavlovian conditioning of stimulus-reward contingencies is sufficient to obtain value-driven attentional capture. Furthermore, value-driven attentional capture can occur following associative reward learning of temporally and spatially task-irrelevant distractors that signal the magnitude of available reward (Experiment 1), and is independent of training spatial shifts of attention towards the reward signaling stimuli (Experiment 2). This confirms and strengthens the idea that Pavlovian reward learning underlies value driven attentional capture.
-
Rajalingham, Rishi; Stacey, Richard Greg; Tsoulfas, Georgios
2014-01-01
To restore movements to paralyzed patients, neural prosthetic systems must accurately decode patients' intentions from neural signals. Despite significant advancements, current systems are unable to restore complex movements. Decoding reward-related signals from the medial intraparietal area (MIP) could enhance prosthetic performance. However, the dynamics of reward sensitivity in MIP is not known. Furthermore, reward-related modulation in premotor areas has been attributed to behavioral confounds. Here we investigated the stability of reward encoding in MIP by assessing the effect of reward history on reward sensitivity. We recorded from neurons in MIP while monkeys performed a delayed-reach task under two reward schedules. In the variable schedule, an equal number of small- and large-rewards trials were randomly interleaved. In the constant schedule, one reward size was delivered for a block of trials. The memory period firing rate of most neurons in response to identical rewards varied according to schedule. Using systems identification tools, we attributed the schedule sensitivity to the dependence of neural activity on the history of reward. We did not find schedule-dependent behavioral changes, suggesting that reward modulates neural activity in MIP. Neural discrimination between rewards was less in the variable than in the constant schedule, degrading our ability to decode reach target and reward simultaneously. The effect of schedule was mitigated by adding Haar wavelet coefficients to the decoding model. This raises the possibility of multiple encoding schemes at different timescales and reinforces the potential utility of reward information for prosthetic performance. PMID:25008408
-
Rajalingham, Rishi; Stacey, Richard Greg; Tsoulfas, Georgios; Musallam, Sam
2014-10-01
To restore movements to paralyzed patients, neural prosthetic systems must accurately decode patients' intentions from neural signals. Despite significant advancements, current systems are unable to restore complex movements. Decoding reward-related signals from the medial intraparietal area (MIP) could enhance prosthetic performance. However, the dynamics of reward sensitivity in MIP is not known. Furthermore, reward-related modulation in premotor areas has been attributed to behavioral confounds. Here we investigated the stability of reward encoding in MIP by assessing the effect of reward history on reward sensitivity. We recorded from neurons in MIP while monkeys performed a delayed-reach task under two reward schedules. In the variable schedule, an equal number of small- and large-rewards trials were randomly interleaved. In the constant schedule, one reward size was delivered for a block of trials. The memory period firing rate of most neurons in response to identical rewards varied according to schedule. Using systems identification tools, we attributed the schedule sensitivity to the dependence of neural activity on the history of reward. We did not find schedule-dependent behavioral changes, suggesting that reward modulates neural activity in MIP. Neural discrimination between rewards was less in the variable than in the constant schedule, degrading our ability to decode reach target and reward simultaneously. The effect of schedule was mitigated by adding Haar wavelet coefficients to the decoding model. This raises the possibility of multiple encoding schemes at different timescales and reinforces the potential utility of reward information for prosthetic performance. Copyright © 2014 the American Physiological Society.
-
Self-choice enhances value in reward-seeking in primates.
Setogawa, Tsuyoshi; Mizuhiki, Takashi; Matsumoto, Narihisa; Akizawa, Fumika; Shidara, Munetaka
2014-03-01
When an individual chooses one item from two or more alternatives, they compare the values of the expected outcomes. The outcome value can be determined by the associated reward amount, the probability of reward, and the workload required to earn the reward. Rational choice theory states that choices are made to maximize rewards over time, and that the same outcome values lead to an equal likelihood of choices. However, the theory does not distinguish between conditions with the same reward value, even when acquired under different circumstances, and does not always accurately describe real behavior. We have found that allowing a monkey to choose a reward schedule endows the schedule with extra value when compared to performance in an identical schedule that is chosen by another agent (a computer here). This behavior is not consistent with pure rational choice theory. Theoretical analysis using a modified temporal-difference learning model showed an enhanced schedule state value by self-choice. These results suggest that an increased reward value underlies the improved performances by self-choice during reward-seeking behavior. Copyright © 2014 Elsevier Ireland Ltd and the Japan Neuroscience Society. All rights reserved.
-
Duffy, Karl J.; Johnson, Steven D.; Peter, Craig I.
2014-01-01
The net effect of pollen production on fecundity in plants can range from negative – when self-pollen interferes with fecundity due to incompatibility mechanisms, to positive – when pollen availability is associated with increased pollinator visitation and fecundity due to its utilization as a reward. We investigated the responses of bees to pollen and nectar rewards, and the effects of these rewards on pollen deposition and fecundity in the hermaphroditic succulent shrub Aloe tenuior. Self-pollinated plants failed to set fruit, but their ovules were regularly penetrated by self-pollen tubes, which uniformly failed to develop into seeds as expected from ovarian self-incompatibility (or strong early inbreeding depression). Bees consistently foraged for pollen during the morning and early afternoon, but switched to nectar in the late afternoon. As a consequence of this differential foraging, we were able to test the relative contribution to fecundity of pollen- versus nectar-collecting flower visitors. We exposed emasculated and intact flowers in either the morning or late afternoon to foraging bees and showed that emasculation reduced pollen deposition by insects in the morning, but had little effect in the afternoon. Despite the potential for self-pollination to result in ovule discounting due to late-acting self-sterility, fecundity was severely reduced in artificially emasculated plants. Although there were temporal fluctuations in reward preference, most bee visits were for pollen rewards. Therefore the benefit of providing pollen that is accessible to bee foragers outweighs any potential costs to fitness in terms of gender interference in this species. PMID:24755611
-
Yu, Xue; Sonuga-Barke, Edmund
2016-07-28
Individuals with ADHD have been shown to prefer smaller sooner over larger later rewards. This has been explained in terms of abnormally steeper discounting of the value of delayed reinforcers. Evidence for this comes from different experimental paradigms. In some, participants experience delay in the laboratory (real-time delay tasks; R-TD), in others they imagine the delay to reinforcers (hypothetical delay tasks; HD). We directly contrasted the performance of 7- to 12-year-old children with ADHD (n = 23) and matched controls (n = 23) on R-TD and HD tasks with monetary rewards. Children with ADHD displayed steeper temporal discounting on the R-TD, but not the HD tasks. These findings suggest that the experience of waiting prior to the delivery of rewards is an important determinant of heightened temporal discounting in ADHD-a finding consistent with models that emphasize the aversive nature of delay for children. © The Author(s) 2016.
-
Heaven can wait. How religion modulates temporal discounting.
Paglieri, Fabio; Borghi, Anna M; Colzato, Lorenza S; Hommel, Bernhard; Scorolli, Claudia
2013-11-01
Evidence suggests that religious systems have specific effects on attentional and action control processes. The present study investigated whether religions also modulate choices that involve higher-order knowledge and the delay of gratification in particular. We tested Dutch Calvinists, Italian Catholics, and Atheists from both countries/cultures using an intertemporal choice task where participants could choose between a small immediate and a larger delayed monetary reward. Based on the Calvinist theory of predestination and the Catholic concept of a cycle of sin-confession-expiation, we predicted a reduced delay tolerance, i.e., higher discount rate, for Italian Catholics than for Dutch Calvinists, and intermediate rates for the two atheist groups. Analyses of discount rates support our hypotheses. We also found a magnitude effect on temporal discounting and faster responses for large than for small rewards across religions and countries/cultures. We conclude that temporal discounting is specifically modulated by religious upbringing rather than by generic cultural differences.
-
ERIC Educational Resources Information Center
Ebert, Jane E. J.
2010-01-01
Temporal discount rates are often poor predictors of behaviors that we expect will be motivated by the future. The current research suggests this may be because conventional discounting measures are poor measures of the motivational value of future rewards. In six studies, I develop motivation-based measures of the present value (PV) of future…
-
Wu, Howard G; Miyamoto, Yohsuke R; Gonzalez Castro, Luis Nicolas; Ölveczky, Bence P; Smith, Maurice A
2014-02-01
Individual differences in motor learning ability are widely acknowledged, yet little is known about the factors that underlie them. Here we explore whether movement-to-movement variability in motor output, a ubiquitous if often unwanted characteristic of motor performance, predicts motor learning ability. Surprisingly, we found that higher levels of task-relevant motor variability predicted faster learning both across individuals and across tasks in two different paradigms, one relying on reward-based learning to shape specific arm movement trajectories and the other relying on error-based learning to adapt movements in novel physical environments. We proceeded to show that training can reshape the temporal structure of motor variability, aligning it with the trained task to improve learning. These results provide experimental support for the importance of action exploration, a key idea from reinforcement learning theory, showing that motor variability facilitates motor learning in humans and that our nervous systems actively regulate it to improve learning.
-
Temporal structure of motor variability is dynamically regulated and predicts motor learning ability
Wu, Howard G; Miyamoto, Yohsuke R; Castro, Luis Nicolas Gonzalez; Ölveczky, Bence P; Smith, Maurice A
2015-01-01
Individual differences in motor learning ability are widely acknowledged, yet little is known about the factors that underlie them. Here we explore whether movement-to-movement variability in motor output, a ubiquitous if often unwanted characteristic of motor performance, predicts motor learning ability. Surprisingly, we found that higher levels of task-relevant motor variability predicted faster learning both across individuals and across tasks in two different paradigms, one relying on reward-based learning to shape specific arm movement trajectories and the other relying on error-based learning to adapt movements in novel physical environments. We proceeded to show that training can reshape the temporal structure of motor variability, aligning it with the trained task to improve learning. These results provide experimental support for the importance of action exploration, a key idea from reinforcement learning theory, showing that motor variability facilitates motor learning in humans and that our nervous systems actively regulate it to improve learning. PMID:24413700
-
Utsumi, Daniel Augusto; Miranda, Mônica Carolina; Muszkat, Mauro
2016-12-30
Temporal Discounting (TD) reflects a tendency to discount a reward more deeply the longer its delivery is delayed. TD tasks and behavioral scales have been used to investigate 'hot' executive functions in ADHD. The present study analyzed TD task performance shown by ADHD and control groups for correlations with emotional self-regulation metrics from two scales, the Behavior Rating Inventory of Executive Functions (BRIEF) and the Child Behavior Checklist (CBCL). Children (ages 8-12) with ADHD (n=25) and controls (n=24) were assessed using material rewards (toys) for three types of task: Hypothetical (H); Hypothetical with temporal expectation (HTE); and Real (R). Between-group differences were found for the HTE task, on which the ADHD group showed a higher rate of discounting their favorite toy over time, especially at 10s and 20s. This was the only task on which performance significantly correlated with BRIEF metrics, thus suggesting associations between impulsivity and low emotional self-regulation, but no task was correlated with CBCL score. The conclusion is that tasks involving toys and HTE in particular may be used to investigate TD in children with ADHD and as a means of evaluating the interface between the reward system and emotional self-regulation. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.
-
Demurie, Ellen; Roeyers, Herbert; Wiersema, Jan R; Sonuga-Barke, Edmund
2016-04-01
Cognitive and motivational factors differentially affect individuals with mental health problems such as ADHD. Here we introduce a new task to disentangle the relative contribution of inhibitory control and reward anticipation on task performance in children with ADHD and/or autism spectrum disorders (ASD). Typically developing children, children with ADHD, ASD, or both disorders worked during separate sessions for monetary or social rewards in go/no-go tasks with varying inhibitory load levels. Participants also completed a monetary temporal discounting (TD) task. As predicted, task performance was sensitive to both the effects of anticipated reward amount and inhibitory load. Reward amount had different effects depending on inhibitory load level. TD correlated with inhibitory control in the ADHD group. The integration of the monetary incentive delay and go/no-go paradigms was successful. Surprisingly, there was no evidence of inhibitory control deficits or altered reward anticipation in the clinical groups. © The Author(s) 2013.
-
Propranolol reduces reference-dependence in intertemporal choice
Lempert, Karolina M.; Lackovic, Sandra F.; Tobe, Russell H.; Glimcher, Paul W.
2017-01-01
Abstract In intertemporal choices between immediate and delayed rewards, people tend to prefer immediate rewards, often even when the delayed reward is larger. This is known as temporal discounting. It has been proposed that this tendency emerges because immediate rewards are more emotionally arousing than delayed rewards. However, in our previous research, we found no evidence for this but instead found that arousal responses (indexed with pupil dilation) in intertemporal choice are context-dependent. Specifically, arousal tracks the subjective value of the more variable reward option in the paradigm, whether it is immediate or delayed. Nevertheless, people tend to choose the less variable option in the choice task. In other words, their choices are reference-dependent and depend on variance in their recent history of offers. This suggests that there may be a causal relationship between reference-dependent choice and arousal, which we investigate here by reducing arousal pharmacologically using propranolol. Here, we show that propranolol reduces reference-dependence, leading to choices that are less influenced by recent history and more internally consistent. PMID:28992268
-
Baker, Travis E; Holroyd, Clay B
2011-04-01
The reinforcement learning theory of the error-related negativity (ERN) holds that the impact of reward signals carried by the midbrain dopamine system modulates activity of the anterior cingulate cortex (ACC), alternatively disinhibiting and inhibiting the ACC following unpredicted error and reward events, respectively. According to a recent formulation of the theory, activity that is intrinsic to the ACC produces a component of the event-related brain potential (ERP) called the N200, and following unpredicted rewards, the N200 is suppressed by extrinsically applied positive dopamine reward signals, resulting in an ERP component called the feedback-ERN (fERN). Here we demonstrate that, despite extensive spatial and temporal overlap between the two ERP components, the functional processes indexed by the N200 (conflict) and the fERN (reward) are dissociable. These results point toward avenues for future investigation. Copyright © 2011 Elsevier B.V. All rights reserved.
-
O'Connor, David A; Rossiter, Sarah; Yücel, Murat; Lubman, Dan I; Hester, Robert
2012-09-01
We examined the neural basis of the capacity to resist an immediately rewarding stimulus in order to obtain a larger delayed reward. This was investigated with a Go/No-go task employing No-go targets that provided two types of reward outcomes. These were contingent on inhibitory control performance: failure to inhibit Reward No-go targets provided a small monetary reward with immediate feedback; while successful inhibitory control resulted in larger rewards with delayed feedback based on the highest number of consecutive inhibitions. We observed faster Go trial responses with maintained levels of inhibition accuracy during the Reward No-go condition compared to a neutral No-go condition. Comparisons between conditions of BOLD activity showed successful inhibitory control over rewarding No-Go targets was associated with hypoactivity in regions previously associated with regulating emotion and inhibitory control, including insula and right inferior frontal gyrus. In addition, regions previously associated with visual processing centers that are modulated as a function of visual attention, namely the left fusiform and right superior temporal gyri, were hypoactive. These findings suggest a role for attentional disengagement as an aid to withholding response over a rewarding stimulus and are consistent with the notion that gratification can be delayed by directing attention away from immediate rewards. Crown Copyright © 2012. Published by Elsevier Inc. All rights reserved.
-
Modeling the Violation of Reward Maximization and Invariance in Reinforcement Schedules
La Camera, Giancarlo; Richmond, Barry J.
2008-01-01
It is often assumed that animals and people adjust their behavior to maximize reward acquisition. In visually cued reinforcement schedules, monkeys make errors in trials that are not immediately rewarded, despite having to repeat error trials. Here we show that error rates are typically smaller in trials equally distant from reward but belonging to longer schedules (referred to as “schedule length effect”). This violates the principles of reward maximization and invariance and cannot be predicted by the standard methods of Reinforcement Learning, such as the method of temporal differences. We develop a heuristic model that accounts for all of the properties of the behavior in the reinforcement schedule task but whose predictions are not different from those of the standard temporal difference model in choice tasks. In the modification of temporal difference learning introduced here, the effect of schedule length emerges spontaneously from the sensitivity to the immediately preceding trial. We also introduce a policy for general Markov Decision Processes, where the decision made at each node is conditioned on the motivation to perform an instrumental action, and show that the application of our model to the reinforcement schedule task and the choice task are special cases of this general theoretical framework. Within this framework, Reinforcement Learning can approach contextual learning with the mixture of empirical findings and principled assumptions that seem to coexist in the best descriptions of animal behavior. As examples, we discuss two phenomena observed in humans that often derive from the violation of the principle of invariance: “framing,” wherein equivalent options are treated differently depending on the context in which they are presented, and the “sunk cost” effect, the greater tendency to continue an endeavor once an investment in money, effort, or time has been made. The schedule length effect might be a manifestation of these phenomena in monkeys. PMID:18688266
-
Modeling the violation of reward maximization and invariance in reinforcement schedules.
La Camera, Giancarlo; Richmond, Barry J
2008-08-08
It is often assumed that animals and people adjust their behavior to maximize reward acquisition. In visually cued reinforcement schedules, monkeys make errors in trials that are not immediately rewarded, despite having to repeat error trials. Here we show that error rates are typically smaller in trials equally distant from reward but belonging to longer schedules (referred to as "schedule length effect"). This violates the principles of reward maximization and invariance and cannot be predicted by the standard methods of Reinforcement Learning, such as the method of temporal differences. We develop a heuristic model that accounts for all of the properties of the behavior in the reinforcement schedule task but whose predictions are not different from those of the standard temporal difference model in choice tasks. In the modification of temporal difference learning introduced here, the effect of schedule length emerges spontaneously from the sensitivity to the immediately preceding trial. We also introduce a policy for general Markov Decision Processes, where the decision made at each node is conditioned on the motivation to perform an instrumental action, and show that the application of our model to the reinforcement schedule task and the choice task are special cases of this general theoretical framework. Within this framework, Reinforcement Learning can approach contextual learning with the mixture of empirical findings and principled assumptions that seem to coexist in the best descriptions of animal behavior. As examples, we discuss two phenomena observed in humans that often derive from the violation of the principle of invariance: "framing," wherein equivalent options are treated differently depending on the context in which they are presented, and the "sunk cost" effect, the greater tendency to continue an endeavor once an investment in money, effort, or time has been made. The schedule length effect might be a manifestation of these phenomena in monkeys.
-
Homeostatic reinforcement learning for integrating reward collection and physiological stability.
Keramati, Mehdi; Gutkin, Boris
2014-12-02
Efficient regulation of internal homeostasis and defending it against perturbations requires adaptive behavioral strategies. However, the computational principles mediating the interaction between homeostatic and associative learning processes remain undefined. Here we use a definition of primary rewards, as outcomes fulfilling physiological needs, to build a normative theory showing how learning motivated behaviors may be modulated by internal states. Within this framework, we mathematically prove that seeking rewards is equivalent to the fundamental objective of physiological stability, defining the notion of physiological rationality of behavior. We further suggest a formal basis for temporal discounting of rewards by showing that discounting motivates animals to follow the shortest path in the space of physiological variables toward the desired setpoint. We also explain how animals learn to act predictively to preclude prospective homeostatic challenges, and several other behavioral patterns. Finally, we suggest a computational role for interaction between hypothalamus and the brain reward system.
-
Brain connectivity reflects human aesthetic responses to music
Sachs, Matthew E.; Ellis, Robert J.; Schlaug, Gottfried
2016-01-01
Abstract Humans uniquely appreciate aesthetics, experiencing pleasurable responses to complex stimuli that confer no clear intrinsic value for survival. However, substantial variability exists in the frequency and specificity of aesthetic responses. While pleasure from aesthetics is attributed to the neural circuitry for reward, what accounts for individual differences in aesthetic reward sensitivity remains unclear. Using a combination of survey data, behavioral and psychophysiological measures and diffusion tensor imaging, we found that white matter connectivity between sensory processing areas in the superior temporal gyrus and emotional and social processing areas in the insula and medial prefrontal cortex explains individual differences in reward sensitivity to music. Our findings provide the first evidence for a neural basis of individual differences in sensory access to the reward system, and suggest that social–emotional communication through the auditory channel may offer an evolutionary basis for music making as an aesthetically rewarding function in humans. PMID:26966157
-
Improving Temporal Cognition by Enhancing Motivation
Avlar, Billur; Kahn, Julia B.; Jensen, Greg; Kandel, Eric R.; Simpson, Eleanor H.; Balsam, Peter D.
2015-01-01
Increasing motivation can positively impact cognitive performance. Here we employed a cognitive timing task that allows us to detect changes in cognitive performance that are not influenced by general activity or arousal factors such as the speed or persistence of responding. This approach allowed us to manipulate motivation using three different methods; molecular/genetic, behavioral and pharmacological. Increased striatal D2Rs resulted in deficits in temporal discrimination. Switching off the transgene improved motivation in earlier studies, and here partially rescued the temporal discrimination deficit. To manipulate motivation behaviorally, we altered reward magnitude and found that increasing reward magnitude improved timing in control mice and partially rescued timing in the transgenic mice. Lastly, we manipulated motivation pharmacologically using a functionally selective 5-HT2C receptor ligand, SB242084, which we previously found to increase incentive motivation. SB242084 improved temporal discrimination in both control and transgenic mice. Thus, while there is a general intuitive belief that motivation can affect cognition, we here provide a direct demonstration that enhancing motivation, in a variety of ways, can be an effective strategy for enhancing temporal cognition. Understanding the interaction of motivation and cognition is of clinical significance since many psychiatric disorders are characterized by deficits in both domains. PMID:26371378
-
Balasubramani, Pragathi P.; Chakravarthy, V. Srinivasa; Ravindran, Balaraman; Moustafa, Ahmed A.
2014-01-01
Although empirical and neural studies show that serotonin (5HT) plays many functional roles in the brain, prior computational models mostly focus on its role in behavioral inhibition. In this study, we present a model of risk based decision making in a modified Reinforcement Learning (RL)-framework. The model depicts the roles of dopamine (DA) and serotonin (5HT) in Basal Ganglia (BG). In this model, the DA signal is represented by the temporal difference error (δ), while the 5HT signal is represented by a parameter (α) that controls risk prediction error. This formulation that accommodates both 5HT and DA reconciles some of the diverse roles of 5HT particularly in connection with the BG system. We apply the model to different experimental paradigms used to study the role of 5HT: (1) Risk-sensitive decision making, where 5HT controls risk assessment, (2) Temporal reward prediction, where 5HT controls time-scale of reward prediction, and (3) Reward/Punishment sensitivity, in which the punishment prediction error depends on 5HT levels. Thus the proposed integrated RL model reconciles several existing theories of 5HT and DA in the BG. PMID:24795614
-
Music models aberrant rule decoding and reward valuation in dementia
Clark, Camilla N; Golden, Hannah L; McCallion, Oliver; Nicholas, Jennifer M; Cohen, Miriam H; Slattery, Catherine F; Paterson, Ross W; Fletcher, Phillip D; Mummery, Catherine J; Rohrer, Jonathan D; Crutch, Sebastian J; Warren, Jason D
2018-01-01
Abstract Aberrant rule- and reward-based processes underpin abnormalities of socio-emotional behaviour in major dementias. However, these processes remain poorly characterized. Here we used music to probe rule decoding and reward valuation in patients with frontotemporal dementia (FTD) syndromes and Alzheimer’s disease (AD) relative to healthy age-matched individuals. We created short melodies that were either harmonically resolved (‘finished’) or unresolved (‘unfinished’); the task was to classify each melody as finished or unfinished (rule processing) and rate its subjective pleasantness (reward valuation). Results were adjusted for elementary pitch and executive processing; neuroanatomical correlates were assessed using voxel-based morphometry. Relative to healthy older controls, patients with behavioural variant FTD showed impairments of both musical rule decoding and reward valuation, while patients with semantic dementia showed impaired reward valuation but intact rule decoding, patients with AD showed impaired rule decoding but intact reward valuation and patients with progressive non-fluent aphasia performed comparably to healthy controls. Grey matter associations with task performance were identified in anterior temporal, medial and lateral orbitofrontal cortices, previously implicated in computing diverse biological and non-biological rules and rewards. The processing of musical rules and reward distils cognitive and neuroanatomical mechanisms relevant to complex socio-emotional dysfunction in major dementias. PMID:29186630
-
Positive autobiographical memory retrieval reduces temporal discounting
Lempert, Karolina M; Speer, Megan E; Delgado, Mauricio R
2017-01-01
Abstract People generally prefer rewards sooner rather than later. This phenomenon, temporal discounting, underlies many societal problems, including addiction and obesity. One way to reduce temporal discounting is to imagine positive future experiences. Since there is overlap in the neural circuitry associated with imagining future experiences and remembering past events, here we investigate whether recalling positive memories can also promote more patient choice. We found that participants were more patient after retrieving positive autobiographical memories, but not when they recalled negative memories. Moreover, individuals were more impulsive after imagining novel positive scenes that were not related to their memories, showing that positive imagery alone does not drive this effect. Activity in the striatum and temporo parietal junction during memory retrieval predicted more patient choice, suggesting that to the extent that memory recall is rewarding and involves perspective-taking, it influences decision-making. Furthermore, representational similarity in the ventromedial prefrontal cortex between memory recall and decision phases correlated with the behavioral effect across participants. Thus, we have identified a novel manipulation for reducing temporal discounting—remembering the positive past—and have begun to characterize the psychological and neural mechanisms behind it. PMID:28655195
-
Neural correlates of reward processing in adults with 22q11 deletion syndrome.
van Duin, Esther D A; Goossens, Liesbet; Hernaus, Dennis; da Silva Alves, Fabiana; Schmitz, Nicole; Schruers, Koen; van Amelsvoort, Therese
2016-01-01
22q11.2 deletion syndrome (22q11DS) is caused by a microdeletion on chromosome 22q11.2 and associated with an increased risk to develop psychosis. The gene coding for catechol-O-methyl-transferase (COMT) is located at the deleted region, resulting in disrupted dopaminergic neurotransmission in 22q11DS, which may contribute to the increased vulnerability for psychosis. A dysfunctional motivational reward system is considered one of the salient features in psychosis and thought to be related to abnormal dopaminergic neurotransmission. The functional anatomy of the brain reward circuitry has not yet been investigated in 22q11DS. This study aims to investigate neural activity during anticipation of reward and loss in adult patients with 22q11DS. We measured blood-oxygen-level dependent (BOLD) activity in 16 patients with 22q11DS and 12 healthy controls during a monetary incentive delay task using a 3T Philips Intera MRI system. Data were analysed using SPM8. During anticipation of reward, the 22q11DS group alone displayed significant activation in bilateral middle frontal and temporal brain regions. Compared to healthy controls, significantly less activation in bilateral cingulate gyrus extending to premotor, primary motor and somatosensory areas was found. During anticipation of loss, the 22q11DS group displayed activity in the left middle frontal gyrus and anterior cingulate cortex, and relative to controls, they showed reduced brain activation in bilateral (pre)cuneus and left posterior cingulate. Within the 22q11DS group, COMT Val hemizygotes displayed more activation compared to Met hemizygotes in right posterior cingulate and bilateral parietal regions during anticipation of reward. During anticipation of loss, COMT Met hemizygotes compared to Val hemizygotes showed more activation in bilateral insula, striatum and left anterior cingulate. This is the first study to investigate reward processing in 22q11DS. Our preliminary results suggest that people with 22q11DS engage a fronto-temporal neural network. Compared to healthy controls, people with 22q11DS primarily displayed reduced activity in medial frontal regions during reward anticipation. COMT hemizygosity affects responsivity of the reward system in this condition. Alterations in reward processing partly underlain by the dopamine system may play a role in susceptibility for psychosis in 22q11DS.
-
Sex-specific enhancement of palatability-driven feeding in adolescent rats
Liu, Angela T.; Murphy, Niall P.; Maidment, Nigel T.; Ostlund, Sean B.
2017-01-01
It has been hypothesized that brain development during adolescence perturbs reward processing in a way that may ultimately contribute to the risky decision making associated with this stage of life, particularly in young males. To investigate potential reward dysfunction during adolescence, Experiment 1 examined palatable fluid intake in rats as a function of age and sex. During a series of twice-weekly test sessions, non-food-deprived rats were given the opportunity to voluntarily consume a highly palatable sweetened condensed milk (SCM) solution. We found that adolescent male, but not female, rats exhibited a pronounced, transient increase in SCM intake (normalized by body weight) that was centered around puberty. Additionally, adult females consumed more SCM than adult males and adolescent females. Using a well-established analytical framework to parse the influences of reward palatability and satiety on the temporal structure of feeding behavior, we found that palatability-driven intake at the outset of the meal was significantly elevated in adolescent males, relative to the other groups. Furthermore, although we found that there were some group differences in the onset of satiety, they were unlikely to contribute to differences in intake. Experiment 2 confirmed that adolescent male rats exhibit elevated palatable fluid consumption, relative to adult males, even when a non-caloric saccharin solution was used as the taste stimulus, demonstrating that these results were unlikely to be related to age-related differences in metabolic need. These findings suggest that elevated palatable food intake during adolescence is sex specific and driven by a fundamental change in reward processing. As adolescent risk taking has been hypothesized as a potential result of hypersensitivity to and overvaluation of appetitive stimuli, individual differences in reward palatability may factor into individual differences in adolescent risky decision making. PMID:28708901
-
Deep brain stimulation of nucleus accumbens region in alcoholism affects reward processing.
Heldmann, Marcus; Berding, Georg; Voges, Jürgen; Bogerts, Bernhard; Galazky, Imke; Müller, Ulf; Baillot, Gunther; Heinze, Hans-Jochen; Münte, Thomas F
2012-01-01
The influence of bilateral deep brain stimulation (DBS) of the nucleus nucleus (NAcc) on the processing of reward in a gambling paradigm was investigated using H(2)[(15)O]-PET (positron emission tomography) in a 38-year-old man treated for severe alcohol addiction. Behavioral data analysis revealed a less risky, more careful choice behavior under active DBS compared to DBS switched off. PET showed win- and loss-related activations in the paracingulate cortex, temporal poles, precuneus and hippocampus under active DBS, brain areas that have been implicated in action monitoring and behavioral control. Except for the temporal pole these activations were not seen when DBS was deactivated. These findings suggest that DBS of the NAcc may act partially by improving behavioral control.
-
A temporal basis for Weber's law in value perception.
Namboodiri, Vijay Mohan K; Mihalas, Stefan; Hussain Shuler, Marshall G
2014-01-01
Weber's law-the observation that the ability to perceive changes in magnitudes of stimuli is proportional to the magnitude-is a widely observed psychophysical phenomenon. It is also believed to underlie the perception of reward magnitudes and the passage of time. Since many ecological theories state that animals attempt to maximize reward rates, errors in the perception of reward magnitudes and delays must affect decision-making. Using an ecological theory of decision-making (TIMERR), we analyze the effect of multiple sources of noise (sensory noise, time estimation noise, and integration noise) on reward magnitude and subjective value perception. We show that the precision of reward magnitude perception is correlated with the precision of time perception and that Weber's law in time estimation can lead to Weber's law in value perception. The strength of this correlation is predicted to depend on the reward history of the animal. Subsequently, we show that sensory integration noise (either alone or in combination with time estimation noise) also leads to Weber's law in reward magnitude perception in an accumulator model, if it has balanced Poisson feedback. We then demonstrate that the noise in subjective value of a delayed reward, due to the combined effect of noise in both the perception of reward magnitude and delay, also abides by Weber's law. Thus, in our theory we prove analytically that the perception of reward magnitude, time, and subjective value change all approximately obey Weber's law.
-
Klein-Flügge, Miriam C; Kennerley, Steven W; Saraiva, Ana C; Penny, Will D; Bestmann, Sven
2015-03-01
There has been considerable interest from the fields of biology, economics, psychology, and ecology about how decision costs decrease the value of rewarding outcomes. For example, formal descriptions of how reward value changes with increasing temporal delays allow for quantifying individual decision preferences, as in animal species populating different habitats, or normal and clinical human populations. Strikingly, it remains largely unclear how humans evaluate rewards when these are tied to energetic costs, despite the surge of interest in the neural basis of effort-guided decision-making and the prevalence of disorders showing a diminished willingness to exert effort (e.g., depression). One common assumption is that effort discounts reward in a similar way to delay. Here we challenge this assumption by formally comparing competing hypotheses about effort and delay discounting. We used a design specifically optimized to compare discounting behavior for both effort and delay over a wide range of decision costs (Experiment 1). We then additionally characterized the profile of effort discounting free of model assumptions (Experiment 2). Contrary to previous reports, in both experiments effort costs devalued reward in a manner opposite to delay, with small devaluations for lower efforts, and progressively larger devaluations for higher effort-levels (concave shape). Bayesian model comparison confirmed that delay-choices were best predicted by a hyperbolic model, with the largest reward devaluations occurring at shorter delays. In contrast, an altogether different relationship was observed for effort-choices, which were best described by a model of inverse sigmoidal shape that is initially concave. Our results provide a novel characterization of human effort discounting behavior and its first dissociation from delay discounting. This enables accurate modelling of cost-benefit decisions, a prerequisite for the investigation of the neural underpinnings of effort-guided choice and for understanding the deficits in clinical disorders characterized by behavioral inactivity.
-
Klein-Flügge, Miriam C.; Kennerley, Steven W.; Saraiva, Ana C.; Penny, Will D.; Bestmann, Sven
2015-01-01
There has been considerable interest from the fields of biology, economics, psychology, and ecology about how decision costs decrease the value of rewarding outcomes. For example, formal descriptions of how reward value changes with increasing temporal delays allow for quantifying individual decision preferences, as in animal species populating different habitats, or normal and clinical human populations. Strikingly, it remains largely unclear how humans evaluate rewards when these are tied to energetic costs, despite the surge of interest in the neural basis of effort-guided decision-making and the prevalence of disorders showing a diminished willingness to exert effort (e.g., depression). One common assumption is that effort discounts reward in a similar way to delay. Here we challenge this assumption by formally comparing competing hypotheses about effort and delay discounting. We used a design specifically optimized to compare discounting behavior for both effort and delay over a wide range of decision costs (Experiment 1). We then additionally characterized the profile of effort discounting free of model assumptions (Experiment 2). Contrary to previous reports, in both experiments effort costs devalued reward in a manner opposite to delay, with small devaluations for lower efforts, and progressively larger devaluations for higher effort-levels (concave shape). Bayesian model comparison confirmed that delay-choices were best predicted by a hyperbolic model, with the largest reward devaluations occurring at shorter delays. In contrast, an altogether different relationship was observed for effort-choices, which were best described by a model of inverse sigmoidal shape that is initially concave. Our results provide a novel characterization of human effort discounting behavior and its first dissociation from delay discounting. This enables accurate modelling of cost-benefit decisions, a prerequisite for the investigation of the neural underpinnings of effort-guided choice and for understanding the deficits in clinical disorders characterized by behavioral inactivity. PMID:25816114
-
Graphic Somatography: Life Writing, Comics, and the Ethics of Care.
DeFalco, Amelia
2016-09-01
This essay considers the ways in which graphic caregiving memoirs complicate the idealizing tendencies of ethics of care philosophy. The medium's "capacious" layering of words, images, temporalities, and perspectives produces "productive tensions. . . The words and images entwine, but never synthesize" (Chute 2010, 5). In graphic memoirs about care, this "capaciousness" allows for quick oscillation between the rewards and struggles of care work, representing ambiguous, even ambivalent attitudes toward care. Graphic memoirs effectively represent multiple perspectives without synthesis, part of a structural and thematic ambivalence that provides a provocative counterpart to the abstract idealism of ethics of care philosophy.
-
D’Amour-Horvat, Valérie; Leyton, Marco
2014-01-01
Increases and decreases in dopamine (DA) transmission have both been suggested to influence reward-related impulse-control. The present literature review suggests that, in laboratory animals, the systemic administration of DA augmenters preferentially increases susceptibility to premature responding; with continued DA transmission, reward approach behaviors are sustained. Decreases in DA transmission, in comparison, diminish the appeal of distal and difficult to obtain rewards, thereby increasing susceptibility to temporal discounting and other forms of impulsive choice. The evidence available in humans is not incompatible with this model but is less extensive. PMID:25566001
-
Van Schuerbeek, Peter; Baeken, Chris; De Raedt, Rudi; De Mey, Johan; Luypaert, Rob
2011-01-31
The psychobiological personality model of Cloninger distinguishes four heritable temperament traits (harm avoidance (HA), novelty seeking (NS), reward dependence (RD) and persistence (P)) and three character traits (self-directedness (SD), cooperativeness (CO) and self-transcendence (ST)) which develop during lifetime. Prior research already showed that individual differences in temperament are reflected in structural variances in specific brain areas. In this study, we used voxel-based morphometry (VBM) to correlate the different temperament and character traits with local gray and white matter volumes (GMV and WMV) in young healthy female volunteers. We found correlations between the temperament traits and GMV and WMV in the frontal, temporal and limbic regions involved in controlling and generating the corresponding behavior as proposed in Cloninger's theory: anxious for HA, impulsive for NS, reward-directed for RD and goal-directed for P. The character traits correlated with GMV and WMV in the frontal, temporal and limbic regions involved in the corresponding cognitive tasks: self-reflection for SD, mentalizing and empathizing with others for CO and religious belief for ST. This study shows that individual variations in brain morphology can be related to the temperament and character dimensions, and lends support to the hypothesis of a neurobiological basis of personality traits. Copyright © 2010 Elsevier B.V. All rights reserved.
-
Reward-related neural responses are dependent on the beneficiary
Güroğlu, Berna; de Water, Erik; Meuwese, Rosa; Koolschijn, P. Cédric; Peper, Jiska S.; Crone, Eveline A.
2014-01-01
Prior studies have suggested that positive social interactions are experienced as rewarding. Yet, it is not well understood how social relationships influence neural responses to other persons’ gains. In this study, we investigated neural responses during a gambling task in which healthy participants (N = 31; 18 females) could win or lose money for themselves, their best friend or a disliked other (antagonist). At the moment of receiving outcome, person-related activity was observed in the dorsal medial prefrontal cortex (dmPFC), precuneus and temporal parietal junction (TPJ), showing higher activity for friends and antagonists than for self, and this activity was independent of outcome. The only region showing an interaction between the person-participants played for and outcome was the ventral striatum. Specifically, the striatum was more active following gains than losses for self and friends, whereas for the antagonist this pattern was reversed. Together, these results show that, in a context with social and reward information, social aspects are processed in brain regions associated with social cognition (mPFC, TPJ), and reward aspects are processed in primary reward areas (striatum). Furthermore, there is an interaction of social and reward information in the striatum, such that reward-related activity was dependent on social relationship. PMID:23720575
-
Reward priming eliminates color-driven affect in perception.
Hu, Kesong
2018-01-03
Brain and behavior evidence suggests that colors have distinct affective properties. Here, we investigated how reward influences color-driven affect in perception. In Experiment 1, we assessed competition between blue and red patches during a temporal-order judgment (TOJ) across a range of stimulus onset asynchronies (SOAs). During the value reinforcement, reward was linked to either blue (version 1) or red (version 2) in the experiment. The same stimuli then served as test ones in the following unrewarded, unspeeded TOJ task. Our analysis showed that blue patches were consistently seen as occurring first, even when objectively appearing 2nd at short SOAs. This accelerated perception of blue over red was disrupted by prior primes related to reward (vs. neutral) but not perceptional (blue vs. red) priming. Experiment 2 replicated the findings of Experiment 1 while uncoupling action and stimulus values. These results are consistent with the blue-approach and red-avoidance motivation hypothesis and highlight an active nature of the association of reward priming and color processing. Together, the present study implies a link between reward and color affect and contributes to the understanding of how reward influences color affect in visual processing.
-
Norman, Luke J; Carlisi, Christina O; Christakou, Anastasia; Chantiluke, Kaylita; Murphy, Clodagh; Simmons, Andrew; Giampietro, Vincent; Brammer, Michael; Mataix-Cols, David; Rubia, Katya
2017-11-30
Both Attention-Deficit/Hyperactivity Disorder (ADHD) and Obsessive-Compulsive Disorder (OCD) are associated with choice impulsivity, i.e. the tendency to prefer smaller immediate rewards over larger delayed rewards. However, the extent to which this impulsivity is mediated by shared or distinct underlying neural mechanisms is unclear. Twenty-six boys with ADHD, 20 boys with OCD and 20 matched controls (aged 12-18) completed an fMRI version of an individually adjusted temporal discounting (TD) task which requires choosing between a variable amount of money now or £100 in one week, one month or one year. Activations to immediate and delayed reward choices were compared between groups using a three-way ANCOVA. ADHD patients had steeper discounting rates on the task relative to controls. OCD patients did not differ from controls or patients with ADHD. Patients with ADHD and OCD showed predominantly shared activation deficits during TD in fronto-striato-insular-cerebellar regions responsible for self-control and temporal foresight, suggesting that choice impulsivity is mediated by overlapping neural dysfunctions in both disorders. OCD patients alone showed dysfunction relative to controls in right orbitofrontal and rostrolateral prefrontal cortex, extending previous findings of abnormalities in these regions in OCD to the domain of choice impulsiveness. Copyright © 2017 The Authors. Published by Elsevier B.V. All rights reserved.
-
Pan, Wei-Xing; Schmidt, Robert; Wickens, Jeffery R; Hyland, Brian I
2005-06-29
Behavioral conditioning of cue-reward pairing results in a shift of midbrain dopamine (DA) cell activity from responding to the reward to responding to the predictive cue. However, the precise time course and mechanism underlying this shift remain unclear. Here, we report a combined single-unit recording and temporal difference (TD) modeling approach to this question. The data from recordings in conscious rats showed that DA cells retain responses to predicted reward after responses to conditioned cues have developed, at least early in training. This contrasts with previous TD models that predict a gradual stepwise shift in latency with responses to rewards lost before responses develop to the conditioned cue. By exploring the TD parameter space, we demonstrate that the persistent reward responses of DA cells during conditioning are only accurately replicated by a TD model with long-lasting eligibility traces (nonzero values for the parameter lambda) and low learning rate (alpha). These physiological constraints for TD parameters suggest that eligibility traces and low per-trial rates of plastic modification may be essential features of neural circuits for reward learning in the brain. Such properties enable rapid but stable initiation of learning when the number of stimulus-reward pairings is limited, conferring significant adaptive advantages in real-world environments.
-
Manning, Joshua; Reynolds, Gretchen; Saygin, Zeynep M; Hofmann, Stefan G; Pollack, Mark; Gabrieli, John D E; Whitfield-Gabrieli, Susan
2015-01-01
We investigated differences in the intrinsic functional brain organization (functional connectivity) of the human reward system between healthy control participants and patients with social anxiety disorder. Functional connectivity was measured in the resting-state via functional magnetic resonance imaging (fMRI). 53 patients with social anxiety disorder and 33 healthy control participants underwent a 6-minute resting-state fMRI scan. Functional connectivity of the reward system was analyzed by calculating whole-brain temporal correlations with a bilateral nucleus accumbens seed and a ventromedial prefrontal cortex seed. Patients with social anxiety disorder, relative to the control group, had (1) decreased functional connectivity between the nucleus accumbens seed and other regions associated with reward, including ventromedial prefrontal cortex; (2) decreased functional connectivity between the ventromedial prefrontal cortex seed and lateral prefrontal regions, including the anterior and dorsolateral prefrontal cortices; and (3) increased functional connectivity between both the nucleus accumbens seed and the ventromedial prefrontal cortex seed with more posterior brain regions, including anterior cingulate cortex. Social anxiety disorder appears to be associated with widespread differences in the functional connectivity of the reward system, including markedly decreased functional connectivity between reward regions and between reward regions and lateral prefrontal cortices, and markedly increased functional connectivity between reward regions and posterior brain regions.
-
Learned value and object perception: Accelerated perception or biased decisions?
Rajsic, Jason; Perera, Harendri; Pratt, Jay
2017-02-01
Learned value is known to bias visual search toward valued stimuli. However, some uncertainty exists regarding the stage of visual processing that is modulated by learned value. Here, we directly tested the effect of learned value on preattentive processing using temporal order judgments. Across four experiments, we imbued some stimuli with high value and some with low value, using a nonmonetary reward task. In Experiment 1, we replicated the value-driven distraction effect, validating our nonmonetary reward task. Experiment 2 showed that high-value stimuli, but not low-value stimuli, exhibit a prior-entry effect. Experiment 3, which reversed the temporal order judgment task (i.e., reporting which stimulus came second), showed no prior-entry effect, indicating that although a response bias may be present for high-value stimuli, they are still reported as appearing earlier. However, Experiment 4, using a simultaneity judgment task, showed no shift in temporal perception. Overall, our results support the conclusion that learned value biases perceptual decisions about valued stimuli without speeding preattentive stimulus processing.
-
Reward Sensitivity for a Palatable Food Reward Peaks During Pubertal Developmental in Rats
Friemel, Chris M.; Spanagel, Rainer; Schneider, Miriam
2010-01-01
Puberty is a critical period for the initiation of drug use and abuse. Because early drug use onset often accounts for a more severe progression of addiction, it is of importance to understand the underlying mechanisms and neurodevelopmental changes during puberty that are contributing to enhanced reward processing in teenagers. The present study investigated the progression of reward sensitivity toward a natural food reward over the whole course of adolescence in male rats (postnatal days 30–90) by monitoring consummatory, motivational behavior and neurobiological correlates of reward. Using a limited-free intake paradigm, consumption of sweetened condensed milk (SCM) was measured repeatedly in adolescent and adult rats. Additionally, early- and mid-pubertal animals were tested in Progressive Ratio responding for SCM and c-fos protein expression in reward-associated brain structures was examined after odor conditioning for SCM. We found a transient increase in SCM consumption and motivational incentive for SCM during puberty. This increased reward sensitivity was most pronounced around mid-puberty. The behavioral findings are paralleled by enhanced c-fos staining in reward-related structures revealing an intensified neuronal response after reward-cue presentation, distinctive for pubertal animals. Taken together, these data indicate an increase in reward sensitivity during adolescence accompanied by enhanced responsiveness of reward-associated brain structures to incentive stimuli, and it seems that both is strongly pronounced around mid-puberty. Therefore, higher reward sensitivity during pubertal maturation might contribute to the enhanced vulnerability of teenagers for the initiation of experimental drug use. PMID:20700386
-
Now for Me, Later for Us? Effects of Group Context on Temporal Discounting
Charlton, Shawn R.; Yi, Richard; Porter, Caitlin; Carter, Anne E.; Bickel, Warren; Rachlin, Howard
2013-01-01
Delayed rewards are less valuable than immediate rewards. This well-established finding has focused almost entirely on individual outcomes. However, are delayed rewards similarly discounted if they are shared by a group? The current article reports on three experiments exploring the effect of group context on delay discounting. Results indicate that discount rates of individual and group rewards were highly correlated, but that respondents were more willing to wait (decreased discounting) for shared outcomes than for individual outcomes. An explanatory model is proposed suggesting that decreased discount rates in group contexts may be due to the way the effects of both delay and social discounting are combined. That is, in a group context, a person values both a future reward (discounted by delay) and a present reward to another person (discounted by the social distance between them). The results are explained by a combined discount function containing a delay factor and a factor representing the social distance between the decision maker and group members. Practical implications of the fact that shared consequences can increase individual self-control are also discussed. PMID:23641123
-
Temporal dynamics of reward anticipation in the human brain.
Zhang, Yuanyuan; Li, Qi; Wang, Zhao; Liu, Xun; Zheng, Ya
2017-09-01
Reward anticipation is a complex process including cue evaluation, motor preparation, and feedback anticipation. The present study investigated whether these psychological processes were dissociable on neural dynamics in terms of incentive valence and approach motivation. We recorded EEG when participants were performing a monetary incentive delay task, and found a cue-P3 during the cue-evaluation stage, a contingent negative variation (CNV) during the motor-preparation stage, and a stimulus-preceding negativity (SPN) during the feedback-anticipation stage. Critically, both the cue-P3 and SPN exhibited an enhanced sensitivity to gain versus loss anticipation, which was not observed for the CNV. Moreover, both the cue-P3 and SPN, instead of the CNV, for gain anticipation selectively predicted the participants' approach motivation as measured in a following effort expenditure for rewards task, particularly when reward uncertainty was maximal. Together, these results indicate that reward anticipation consists of several sub-stages, each with distinct functional significance, thus providing implications for neuropsychiatric diseases characterized by dysfunction in anticipatory reward processing. Copyright © 2017 Elsevier B.V. All rights reserved.
-
Sellitto, Manuela; Ciaramelli, Elisa; Mattioli, Flavia; di Pellegrino, Giuseppe
2016-01-01
During intertemporal choice, humans tend to prefer small-sooner rewards over larger-delayed rewards, reflecting temporal discounting (TD) of delayed outcomes. Functional neuroimaging (fMRI) evidence has implicated the insular cortex in time-sensitive decisions, yet it is not clear whether activity in this brain region is crucial for, or merely associated with, TD behavior. Here, patients with damage to the insula (Insular patients), control patients with lesions outside the insula, and healthy individuals chose between smaller-sooner and larger-later monetary rewards. Insular patients were less sensitive to sooner rewards than were the control groups, exhibiting reduced TD. A Voxel-based Lesion-Symptom Mapping (VLSM) analysis confirmed a statistically significant association between insular damage and reduced TD. These results indicate that the insular cortex is crucial for intertemporal choice. We suggest that he insula may be necessary to anticipate the bodily/emotional effects of receiving rewards at different delays, influencing the computation of their incentive value. Devoid of such input, insular patients’ choices would be governed by a heuristic of quantity, allowing patients to wait for larger options. PMID:26793084
-
Völlm, Birgit; Richardson, Paul; McKie, Shane; Elliott, Rebecca; Dolan, Mairead; Deakin, Bill
2007-11-15
Decision making is guided by the likely consequences of behavioural choices. Neuronal correlates of financial reward have been described in a number of functional imaging studies in humans. Areas implicated in reward include ventral striatum, dopaminergic midbrain, amygdala and orbitofrontal cortex. Response to loss has not been as extensively studied but may involve prefrontal and medial temporal cortices. It has been proposed that increased sensitivity to reward and reduced sensitivity to punishment underlie some of the psychopathology in impulsive personality disordered individuals. However, few imaging studies using reinforcement tasks have been conducted in this group. In this fMRI study, we investigate the effects of positive (monetary reward) and negative (monetary loss) outcomes on BOLD responses in two target selection tasks. The experimental group comprised eight people with Cluster B (antisocial and borderline) personality disorder, whilst the control group contained fourteen healthy participants. A key finding was the absence of prefrontal responses and reduced BOLD signal in the subcortical reward system in the PD group during positive reinforcement. Impulsivity scores correlated negatively with prefrontal responses in the PD but not the control group during both, reward and loss. Our results suggest dysfunctional responses to rewarding and aversive stimuli in Cluster B personality disordered individuals but do not support the notion of hypersensitivity to reward and hyposensitivity to loss.
-
O'Mara, Shane M; Sanchez-Vives, Maria V; Brotons-Mas, Jorge R; O'Hare, Eugene
2009-08-01
The subiculum is in a pivotal position governing the output of the hippocampal formation. Despite this, it is a rather under-explored and sometimes ignored structure. Here, we discuss recent data indicating that the subiculum participates in a wide range of neurocognitive functions and processes. Some of the functions of subiculum are relatively well-known-these include providing a relatively coarse representation of space and participating in, and supporting certain aspects of, memory (particularly in the dynamic bridging of temporal intervals). The subiculum also participates in a wide variety of other neurocognitive functions too, however. Much less well-known are roles for the subiculum, and particularly the ventral subiculum, in the response to fear, stress and anxiety, and in the generation of motivated behaviour (particularly the behaviour that underlies drug addiction and the response to reward). There is an emerging suggestion that the subiculum participates in the temporal control of behaviour. It is notable that these latter findings have emerged from a consideration of instrumental behaviour using operant techniques; it may well be the case that the use of the watermaze or similar spatial tasks to assess subicular function (on the presumption that its functions are very similar to the hippocampus proper) has obscured rather than revealed neurocognitive functions of subiculum. The anatomy of subiculum suggests it participates in a rather subtle fashion in a very broad range of functions, rather than in a relatively more isolated fashion in a narrower range of functions, as might be the case for "earlier" components of hippocampal circuitry, such as the CA1 and CA3 subfields. Overall, there appears to a strong dorso-ventral segregation of function within subiculum, with the dorsal subiculum relatively more concerned with space and memory, and the ventral hippocampus concerned with stress, anxiety and reward. Finally, it may be the case that the whole subiculum participates in the temporal control of reinforced behaviour, although further experimentation is required to clarify this hypothesis.
-
Category Learning in the Brain
Seger, Carol A.; Miller, Earl K.
2013-01-01
The ability to group items and events into functional categories is a fundamental characteristic of sophisticated thought. It is subserved by plasticity in many neural systems, including neocortical regions (sensory, prefrontal, parietal, and motor cortex), the medial temporal lobe, the basal ganglia, and midbrain dopaminergic systems. These systems interact during category learning. Corticostriatal loops may mediate recursive, bootstrapping interactions between fast reward-gated plasticity in the basal ganglia and slow reward-shaded plasticity in the cortex. This can provide a balance between acquisition of details of experiences and generalization across them. Interactions between the corticostriatal loops can integrate perceptual, response, and feedback-related aspects of the task and mediate the shift from novice to skilled performance. The basal ganglia and medial temporal lobe interact competitively or cooperatively, depending on the demands of the learning task. PMID:20572771
-
Mutual Influence of Reward Anticipation and Emotion on Brain Activity during Memory Retrieval.
Yan, Chunping; Liu, Fang; Li, Yunyun; Zhang, Qin; Cui, Lixia
2017-01-01
Previous studies on the joint effect of reward motivation and emotion on memory retrieval have obtained inconsistent results. Furthermore, whether and how any such joint effect might vary over time remains unclear too. Accordingly, using the event-related potential (ERP) measurement of high temporal resolution, our study investigates the cognitive and brain mechanisms of monetary reward and emotion affecting the retrieval processes of episodic memory. Twenty undergraduate and graduate students participated in the research, and our study's behavioral results indicated that reward (relative to no reward) and negative emotion (relative to positive and neutral emotion) significantly improved recognition performance. The ERP results showed that there were significant interactions between monetary reward and emotion on memory retrieval, and the reward effects of positive, neutral, and negative memory occurred at varied intervals in mean amplitude. The reward effect of positive memory appeared relatively early, at 260-330 ms after the stimulus onset in the frontal-frontocentral area, at 260-500 ms in the centroparietal-parietal area and at 500-700 ms in the frontocentral area. However, the reward effects of neutral and negative memory occurred relatively later, and that of negative memory appeared at 500-700 ms in the frontocentral and centroparietal area and that of neutral memory was at 500-700 ms in the frontocentral and centroparietal-parietal area. Meanwhile, significant FN400 old/new effects were observed in the negative and rewarded positive items, and the old/new effects of negative items appeared earlier at FN400 than positive items. Also, significant late positive component (LPC) old/new effects were found in the positive, negative, and rewarded neutral items. These results suggest that, monetary reward and negative emotion significantly improved recognition performance, and there was a mutual influence between reward and emotion on brain activity during memory retrieval.
-
Brain connectivity reflects human aesthetic responses to music.
Sachs, Matthew E; Ellis, Robert J; Schlaug, Gottfried; Loui, Psyche
2016-06-01
Humans uniquely appreciate aesthetics, experiencing pleasurable responses to complex stimuli that confer no clear intrinsic value for survival. However, substantial variability exists in the frequency and specificity of aesthetic responses. While pleasure from aesthetics is attributed to the neural circuitry for reward, what accounts for individual differences in aesthetic reward sensitivity remains unclear. Using a combination of survey data, behavioral and psychophysiological measures and diffusion tensor imaging, we found that white matter connectivity between sensory processing areas in the superior temporal gyrus and emotional and social processing areas in the insula and medial prefrontal cortex explains individual differences in reward sensitivity to music. Our findings provide the first evidence for a neural basis of individual differences in sensory access to the reward system, and suggest that social-emotional communication through the auditory channel may offer an evolutionary basis for music making as an aesthetically rewarding function in humans. © The Author (2016). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
The basolateral amygdala in reward learning and addiction
Wassum, Kate M.; Izquierdo, Alicia
2015-01-01
Sophisticated behavioral paradigms partnered with the emergence of increasingly selective techniques to target the basolateral amygdala (BLA) have resulted in an enhanced understanding of the role of this nucleus in learning and using reward information. Due to the wide variety of behavioral approaches many questions remain on the circumscribed role of BLA in appetitive behavior. In this review, we integrate conclusions of BLA function in reward-related behavior using traditional interference techniques (lesion, pharmacological inactivation) with those using newer methodological approaches in experimental animals that allow in vivo manipulation of cell type-specific populations and neural recordings. Secondly, from a review of appetitive behavioral tasks in rodents and monkeys and recent computational models of reward procurement, we derive evidence for BLA as a neural integrator of reward value, history, and cost parameters. Taken together, BLA codes specific and temporally dynamic outcome representations in a distributed network to orchestrate adaptive responses. We provide evidence that experiences with opiates and psychostimulants alter these outcome representations in BLA, resulting in long-term modified action. PMID:26341938
-
Homeostatic reinforcement learning for integrating reward collection and physiological stability
Keramati, Mehdi; Gutkin, Boris
2014-01-01
Efficient regulation of internal homeostasis and defending it against perturbations requires adaptive behavioral strategies. However, the computational principles mediating the interaction between homeostatic and associative learning processes remain undefined. Here we use a definition of primary rewards, as outcomes fulfilling physiological needs, to build a normative theory showing how learning motivated behaviors may be modulated by internal states. Within this framework, we mathematically prove that seeking rewards is equivalent to the fundamental objective of physiological stability, defining the notion of physiological rationality of behavior. We further suggest a formal basis for temporal discounting of rewards by showing that discounting motivates animals to follow the shortest path in the space of physiological variables toward the desired setpoint. We also explain how animals learn to act predictively to preclude prospective homeostatic challenges, and several other behavioral patterns. Finally, we suggest a computational role for interaction between hypothalamus and the brain reward system. DOI: http://dx.doi.org/10.7554/eLife.04811.001 PMID:25457346
-
Temporal discounting in major depressive disorder.
Pulcu, E; Trotter, P D; Thomas, E J; McFarquhar, M; Juhasz, G; Sahakian, B J; Deakin, J F W; Zahn, R; Anderson, I M; Elliott, R
2014-07-01
Major depressive disorder (MDD) is associated with abnormalities in financial reward processing. Previous research suggests that patients with MDD show reduced sensitivity to frequency of financial rewards. However, there is a lack of conclusive evidence from studies investigating the evaluation of financial rewards over time, an important aspect of reward processing that influences the way people plan long-term investments. Beck's cognitive model posits that patients with MDD hold a negative view of the future that may influence the amount of resources patients are willing to invest into their future selves. We administered a delay discounting task to 82 participants: 29 healthy controls, 29 unmedicated participants with fully remitted MDD (rMDD) and 24 participants with current MDD (11 on medication). Patients with current MDD, relative to remitted patients and healthy subjects, discounted large-sized future rewards at a significantly higher rate and were insensitive to changes in reward size from medium to large. There was a main effect of clinical group on discounting rates for large-sized rewards, and discounting rates for large-sized rewards correlated with severity of depressive symptoms, particularly hopelessness. Higher discounting of delayed rewards in MDD seems to be state dependent and may be a reflection of depressive symptoms, specifically hopelessness. Discounting distant rewards at a higher rate means that patients are more likely to choose immediate financial options. Such impairments related to long-term investment planning may be important for understanding value-based decision making in MDD, and contribute to ongoing functional impairment.
-
Separate and overlapping brain areas encode subjective value during delay and effort discounting.
Massar, Stijn A A; Libedinsky, Camilo; Weiyan, Chee; Huettel, Scott A; Chee, Michael W L
2015-10-15
Making decisions about rewards that involve delay or effort requires the integration of value and cost information. The brain areas recruited in this integration have been well characterized for delay discounting. However only a few studies have investigated how effort costs are integrated into value signals to eventually determine choice. In contrast to previous studies that have evaluated fMRI signals related to physical effort, we used a task that focused on cognitive effort. Participants discounted the value of delayed and effortful rewards. The value of cognitively effortful rewards was represented in the anterior portion of the inferior frontal gyrus and dorsolateral prefrontal cortex. Additionally, the value of the chosen option was encoded in the anterior cingulate cortex, caudate, and cerebellum. While most brain regions showed no significant dissociation between effort discounting and delay discounting, the ACC was significantly more activated in effort compared to delay discounting tasks. Finally, overlapping regions within the right orbitofrontal cortex and lateral temporal and parietal cortices encoded the value of the chosen option during both delay and effort discounting tasks. These results indicate that encoding of rewards discounted by cognitive effort and delay involves partially dissociable brain areas, but a common representation of chosen value is present in the orbitofrontal, temporal and parietal cortices. Copyright © 2015 The Authors. Published by Elsevier Inc. All rights reserved.
-
How decisions emerge: action dynamics in intertemporal decision making.
Dshemuchadse, Maja; Scherbaum, Stefan; Goschke, Thomas
2013-02-01
In intertemporal decision making, individuals prefer smaller rewards delivered sooner over larger rewards delivered later, often to an extent that seems irrational from an economical perspective. This behavior has been attributed to a lack of self-control and reflection, the nonlinearity of human time perception, and several other sources. Although an increasing number of models propose different mathematical descriptions of temporal discounting, the dynamics of the decision process behind temporal discounting are much less clear. In this study, we obtained further insights into the mechanisms of intertemporal decisions by observing choice action dynamics via a novel combination of continuously recorded mouse movements and a multiple regression approach. Participants had to choose between two hypothetical options (sooner/smaller vs. later/larger) by moving the mouse cursor from the bottom of the screen either to the top left or to the top right. We observed less direct mouse movements when participants chose later/larger rewards, indicating that participants had to overcome the attraction of the sooner/smaller reward first. Additionally, our results suggest that framing time information differently changes the weighting of value. We conclude that using a continuous process-oriented approach could further advance the understanding of intertemporal choice beyond the identification of the best fitted mathematical description of the discounting function by uncovering the way intertemporal decisions are performed. 2013 APA, all rights reserved
-
Positive autobiographical memory retrieval reduces temporal discounting.
Lempert, Karolina M; Speer, Megan E; Delgado, Mauricio R; Phelps, Elizabeth A
2017-10-01
People generally prefer rewards sooner rather than later. This phenomenon, temporal discounting, underlies many societal problems, including addiction and obesity. One way to reduce temporal discounting is to imagine positive future experiences. Since there is overlap in the neural circuitry associated with imagining future experiences and remembering past events, here we investigate whether recalling positive memories can also promote more patient choice. We found that participants were more patient after retrieving positive autobiographical memories, but not when they recalled negative memories. Moreover, individuals were more impulsive after imagining novel positive scenes that were not related to their memories, showing that positive imagery alone does not drive this effect. Activity in the striatum and temporo parietal junction during memory retrieval predicted more patient choice, suggesting that to the extent that memory recall is rewarding and involves perspective-taking, it influences decision-making. Furthermore, representational similarity in the ventromedial prefrontal cortex between memory recall and decision phases correlated with the behavioral effect across participants. Thus, we have identified a novel manipulation for reducing temporal discounting-remembering the positive past-and have begun to characterize the psychological and neural mechanisms behind it. © The Author (2017). Published by Oxford University Press.
-
Dopamine reward prediction errors reflect hidden state inference across time
Starkweather, Clara Kwon; Babayan, Benedicte M.; Uchida, Naoshige; Gershman, Samuel J.
2017-01-01
Midbrain dopamine neurons signal reward prediction error (RPE), or actual minus expected reward. The temporal difference (TD) learning model has been a cornerstone in understanding how dopamine RPEs could drive associative learning. Classically, TD learning imparts value to features that serially track elapsed time relative to observable stimuli. In the real world, however, sensory stimuli provide ambiguous information about the hidden state of the environment, leading to the proposal that TD learning might instead compute a value signal based on an inferred distribution of hidden states (a ‘belief state’). In this work, we asked whether dopaminergic signaling supports a TD learning framework that operates over hidden states. We found that dopamine signaling exhibited a striking difference between two tasks that differed only with respect to whether reward was delivered deterministically. Our results favor an associative learning rule that combines cached values with hidden state inference. PMID:28263301
-
Murawski, Carsten; Harris, Philip G; Bode, Stefan; Domínguez D, Juan F; Egan, Gary F
2012-01-01
Human decision-making is driven by subjective values assigned to alternative choice options. These valuations are based on reward cues. It is unknown, however, whether complex reward cues, such as brand logos, may bias the neural encoding of subjective value in unrelated decisions. In this functional magnetic resonance imaging (fMRI) study, we subliminally presented brand logos preceding intertemporal choices. We demonstrated that priming biased participants' preferences towards more immediate rewards in the subsequent temporal discounting task. This was associated with modulations of the neural encoding of subjective values of choice options in a network of brain regions, including but not restricted to medial prefrontal cortex. Our findings demonstrate the general susceptibility of the human decision making system to apparently incidental contextual information. We conclude that the brain incorporates seemingly unrelated value information that modifies decision making outside the decision-maker's awareness.
-
Led into Temptation? Rewarding Brand Logos Bias the Neural Encoding of Incidental Economic Decisions
Murawski, Carsten; Harris, Philip G.; Bode, Stefan; Domínguez D., Juan F.; Egan, Gary F.
2012-01-01
Human decision-making is driven by subjective values assigned to alternative choice options. These valuations are based on reward cues. It is unknown, however, whether complex reward cues, such as brand logos, may bias the neural encoding of subjective value in unrelated decisions. In this functional magnetic resonance imaging (fMRI) study, we subliminally presented brand logos preceding intertemporal choices. We demonstrated that priming biased participants' preferences towards more immediate rewards in the subsequent temporal discounting task. This was associated with modulations of the neural encoding of subjective values of choice options in a network of brain regions, including but not restricted to medial prefrontal cortex. Our findings demonstrate the general susceptibility of the human decision making system to apparently incidental contextual information. We conclude that the brain incorporates seemingly unrelated value information that modifies decision making outside the decision-maker's awareness. PMID:22479547
-
Magnetoencephalographic imaging of deep corticostriatal network activity during a rewards paradigm.
Kanal, Eliezer Y; Sun, Mingui; Ozkurt, Tolga E; Jia, Wenyan; Sclabassi, Robert
2009-01-01
The human rewards network is a complex system spanning both cortical and subcortical regions. While much is known about the functions of the various components of the network, research on the behavior of the network as a whole has been stymied due to an inability to detect signals at a high enough temporal resolution from both superficial and deep network components simultaneously. In this paper, we describe the application of magnetoencephalographic imaging (MEG) combined with advanced signal processing techniques to this problem. Using data collected while subjects performed a rewards-related gambling paradigm demonstrated to activate the rewards network, we were able to identify neural signals which correspond to deep network activity. We also show that this signal was not observable prior to filtration. These results suggest that MEG imaging may be a viable tool for the detection of deep neural activity.
-
Dopamine reward prediction errors reflect hidden-state inference across time.
Starkweather, Clara Kwon; Babayan, Benedicte M; Uchida, Naoshige; Gershman, Samuel J
2017-04-01
Midbrain dopamine neurons signal reward prediction error (RPE), or actual minus expected reward. The temporal difference (TD) learning model has been a cornerstone in understanding how dopamine RPEs could drive associative learning. Classically, TD learning imparts value to features that serially track elapsed time relative to observable stimuli. In the real world, however, sensory stimuli provide ambiguous information about the hidden state of the environment, leading to the proposal that TD learning might instead compute a value signal based on an inferred distribution of hidden states (a 'belief state'). Here we asked whether dopaminergic signaling supports a TD learning framework that operates over hidden states. We found that dopamine signaling showed a notable difference between two tasks that differed only with respect to whether reward was delivered in a deterministic manner. Our results favor an associative learning rule that combines cached values with hidden-state inference.
-
Dopamine, reward learning, and active inference
FitzGerald, Thomas H. B.; Dolan, Raymond J.; Friston, Karl
2015-01-01
Temporal difference learning models propose phasic dopamine signaling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behavior. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings. PMID:26581305
-
Dopamine, reward learning, and active inference.
FitzGerald, Thomas H B; Dolan, Raymond J; Friston, Karl
2015-01-01
Temporal difference learning models propose phasic dopamine signaling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behavior. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.
-
Brain reward region responsivity of adolescents with and without parental substance use disorders.
Stice, Eric; Yokum, Sonja
2014-09-01
The present study tested the competing hypotheses that adolescents at risk for future substance abuse and dependence by virtue of parental substance use disorders show either weaker or stronger responsivity of brain regions implicated in reward relative to youth without parental history of substance use disorders. Adolescents (n = 52) matched on demographics with and without parental substance use disorders, as determined by diagnostic interviews, who denied substance use in the past year were compared on functional MRI (fMRI) paradigms assessing neural response to receipt and anticipated receipt of monetary and food reward. Parental-history-positive versus -negative adolescents showed greater activation in the left dorsolateral prefrontal cortex and bilateral putamen, and less activation in the fusiform gyrus and inferior temporal gyrus in response to anticipating winning money, as well as greater activation in the left midbrain and right paracentral lobule, and less activation in the right middle frontal gyrus in response to milkshake receipt. Results indicate that adolescents at risk for future onset of substance use disorders show elevated responsivity of brain regions implicated in reward, extending results from 2 smaller prior studies that found that individuals with versus without parental alcohol use disorders showed greater reward region response to anticipated monetary reward and pictures of alcohol. Collectively, results provide support for the reward surfeit model of substance use disorders, rather than the reward deficit model.
-
Kim, Ji-Eun; Son, Jung-Woo; Choi, Won-Hee; Kim, Yeoung-Rang; Oh, Jong-Hyun; Lee, Seungbok; Kim, Jang-Kyu
2014-06-01
This study aimed to examine differences in brain activation for various types of reward and feedback in adolescent Internet addicts (AIA) and normal adolescents (NA) using functional magnetic resonance imaging (fMRI). AIA (n = 15) and NA (n = 15) underwent fMRI while performing easy tasks for which performance feedback (PF), social reward (SR) (such as compliments), or monetary reward (MR) was given. Using the no reward (NR) condition, three types of contrasts (PF-NR, SR-NR, and MR-NR) were analyzed. In NA, we observed activation in the reward-related subcortical system, self-related brain region, and other brain areas for the three contrasts, but these brain areas showed almost no activation in AIA. Instead, AIA showed significant activation in the dorsolateral prefrontal cortex for the PF-NR contrast and the negative correlation was found between the level of activation in the left superior temporal gyrus (BA 22) and the duration of Internet game use per day in AIA. These findings suggest that AIA show reduced levels of self-related brain activation and decreased reward sensitivity irrespective of the type of reward and feedback. AIA may be only sensitive to error monitoring regardless of positive feelings, such as sense of satisfaction or achievement. © 2014 The Authors. Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology.
-
Chiew, Kimberly S.; Braver, Todd S.
2013-01-01
Motivational manipulations, such as the presence of performance-contingent reward incentives, can have substantial influences on cognitive control. Previous evidence suggests that reward incentives may enhance cognitive performance specifically through increased preparatory, or proactive, control processes. The present study examined reward influences on cognitive control dynamics in the AX-Continuous Performance Task (AX-CPT), using high-resolution pupillometry. In the AX-CPT, contextual cues must be actively maintained over a delay in order to appropriately respond to ambiguous target probes. A key feature of the task is that it permits dissociable characterization of preparatory, proactive control processes (i.e., utilization of context) and reactive control processes (i.e., target-evoked interference resolution). Task performance profiles suggested that reward incentives enhanced proactive control (context utilization). Critically, pupil dilation was also increased on reward incentive trials during context maintenance periods, suggesting trial-specific shifts in proactive control, particularly when context cues indicated the need to overcome the dominant target response bias. Reward incentives had both transient (i.e., trial-by-trial) and sustained (i.e., block-based) effects on pupil dilation, which may reflect distinct underlying processes. The transient pupillary effects were present even when comparing against trials matched in task performance, suggesting a unique motivational influence of reward incentives. These results suggest that pupillometry may be a useful technique for investigating reward motivational signals and their dynamic influence on cognitive control. PMID:23372557
-
Reward-related neural responses are dependent on the beneficiary.
Braams, Barbara R; Güroğlu, Berna; de Water, Erik; Meuwese, Rosa; Koolschijn, P Cédric; Peper, Jiska S; Crone, Eveline A
2014-07-01
Prior studies have suggested that positive social interactions are experienced as rewarding. Yet, it is not well understood how social relationships influence neural responses to other persons' gains. In this study, we investigated neural responses during a gambling task in which healthy participants (N = 31; 18 females) could win or lose money for themselves, their best friend or a disliked other (antagonist). At the moment of receiving outcome, person-related activity was observed in the dorsal medial prefrontal cortex (dmPFC), precuneus and temporal parietal junction (TPJ), showing higher activity for friends and antagonists than for self, and this activity was independent of outcome. The only region showing an interaction between the person-participants played for and outcome was the ventral striatum. Specifically, the striatum was more active following gains than losses for self and friends, whereas for the antagonist this pattern was reversed. Together, these results show that, in a context with social and reward information, social aspects are processed in brain regions associated with social cognition (mPFC, TPJ), and reward aspects are processed in primary reward areas (striatum). Furthermore, there is an interaction of social and reward information in the striatum, such that reward-related activity was dependent on social relationship. © The Author (2013). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
Fronto-temporal white matter connectivity predicts reversal learning errors
Alm, Kylie H.; Rolheiser, Tyler; Mohamed, Feroze B.; Olson, Ingrid R.
2015-01-01
Each day, we make hundreds of decisions. In some instances, these decisions are guided by our innate needs; in other instances they are guided by memory. Probabilistic reversal learning tasks exemplify the close relationship between decision making and memory, as subjects are exposed to repeated pairings of a stimulus choice with a reward or punishment outcome. After stimulus–outcome associations have been learned, the associated reward contingencies are reversed, and participants are not immediately aware of this reversal. Individual differences in the tendency to choose the previously rewarded stimulus reveal differences in the tendency to make poorly considered, inflexible choices. Lesion studies have strongly linked reversal learning performance to the functioning of the orbitofrontal cortex, the hippocampus, and in some instances, the amygdala. Here, we asked whether individual differences in the microstructure of the uncinate fasciculus, a white matter tract that connects anterior and medial temporal lobe regions to the orbitofrontal cortex, predict reversal learning performance. Diffusion tensor imaging and behavioral paradigms were used to examine this relationship in 33 healthy young adults. The results of tractography revealed a significant negative relationship between reversal learning performance and uncinate axial diffusivity, but no such relationship was demonstrated in a control tract, the inferior longitudinal fasciculus. Our findings suggest that the uncinate might serve to integrate associations stored in the anterior and medial temporal lobes with expectations about expected value based on feedback history, computed in the orbitofrontal cortex. PMID:26150776
-
Segarra, Nuria; Metastasio, Antonio; Ziauddeen, Hisham; Spencer, Jennifer; Reinders, Niels R; Dudas, Robert B; Arrondo, Gonzalo; Robbins, Trevor W; Clark, Luke; Fletcher, Paul C; Murray, Graham K
2016-07-01
Alterations in reward processes may underlie motivational and anhedonic symptoms in depression and schizophrenia. However it remains unclear whether these alterations are disorder-specific or shared, and whether they clearly relate to symptom generation or not. We studied brain responses to unexpected rewards during a simulated slot-machine game in 24 patients with depression, 21 patients with schizophrenia, and 21 healthy controls using functional magnetic resonance imaging. We investigated relationships between brain activation, task-related motivation, and questionnaire rated anhedonia. There was reduced activation in the orbitofrontal cortex, ventral striatum, inferior temporal gyrus, and occipital cortex in both depression and schizophrenia in comparison with healthy participants during receipt of unexpected reward. In the medial prefrontal cortex both patient groups showed reduced activation, with activation significantly more abnormal in schizophrenia than depression. Anterior cingulate and medial frontal cortical activation predicted task-related motivation, which in turn predicted anhedonia severity in schizophrenia. Our findings provide evidence for overlapping hypofunction in ventral striatal and orbitofrontal regions in depression and schizophrenia during unexpected reward receipt, and for a relationship between unexpected reward processing in the medial prefrontal cortex and the generation of motivational states.
-
Segarra, Nuria; Metastasio, Antonio; Ziauddeen, Hisham; Spencer, Jennifer; Reinders, Niels R; Dudas, Robert B; Arrondo, Gonzalo; Robbins, Trevor W; Clark, Luke; Fletcher, Paul C; Murray, Graham K
2016-01-01
Alterations in reward processes may underlie motivational and anhedonic symptoms in depression and schizophrenia. However it remains unclear whether these alterations are disorder-specific or shared, and whether they clearly relate to symptom generation or not. We studied brain responses to unexpected rewards during a simulated slot-machine game in 24 patients with depression, 21 patients with schizophrenia, and 21 healthy controls using functional magnetic resonance imaging. We investigated relationships between brain activation, task-related motivation, and questionnaire rated anhedonia. There was reduced activation in the orbitofrontal cortex, ventral striatum, inferior temporal gyrus, and occipital cortex in both depression and schizophrenia in comparison with healthy participants during receipt of unexpected reward. In the medial prefrontal cortex both patient groups showed reduced activation, with activation significantly more abnormal in schizophrenia than depression. Anterior cingulate and medial frontal cortical activation predicted task-related motivation, which in turn predicted anhedonia severity in schizophrenia. Our findings provide evidence for overlapping hypofunction in ventral striatal and orbitofrontal regions in depression and schizophrenia during unexpected reward receipt, and for a relationship between unexpected reward processing in the medial prefrontal cortex and the generation of motivational states. PMID:26708106
-
Exposure to sucrose during periods of withdrawal does not reduce cocaine-seeking behavior in rats.
Nicolas, Céline; Lafay-Chebassier, Claire; Solinas, Marcello
2016-03-21
Concomitant access to drugs of abuse and alternative rewards such as sucrose has been shown to decrease addiction-related behaviors in animals. Here we investigated whether access to sucrose during abstinence in contexts that are temporally and physically distinct from drug-related contexts could reduce subsequent drug seeking. In addition, we investigated whether a history of cocaine self-administration would alter the rewarding effects of sucrose. Rats self-administered cocaine for ten sessions, while yoked-saline rats received only saline injections, and then we subjected them to a 30-day withdrawal period during which they had access to water and sucrose continuously or intermittently according to a schedule that induces binge-drinking behavior. At the end of the withdrawal period, rats were tested for cocaine seeking behavior during a single 6 h session. We found that exposure to cocaine increased sucrose consumption only when rats had intermittent access to sucrose, but exposure to sucrose did not alter drug seeking regardless of the schedule of access. These results suggest that exposure to cocaine cross-sensitizes to the rewarding effects of sucrose, but exposure to sucrose during abstinence, temporally and physically distinct from drug-related environments, does not to reduce drug seeking.
-
Metabolic and reward feeding synchronises the rhythmic brain.
Challet, Etienne; Mendoza, Jorge
2010-07-01
Daily brain rhythmicity, which controls the sleep-wake cycle and neuroendocrine functions, is generated by an endogenous circadian timing system. Within the multi-oscillatory circadian network, a master clock is located in the suprachiasmatic nuclei of the hypothalamus, whose main synchroniser (Zeitgeber) is light. In contrast, imposed meal times and temporally restricted feeding are potent synchronisers for secondary clocks in peripheral organs such as the liver and in brain regions, although not for the suprachiasmatic nuclei. Even when animals are exposed to a light-dark cycle, timed calorie restriction (i.e. when only a hypocaloric diet is given every day) is a synchroniser powerful enough to modify the suprachiasmatic clockwork and increase the synchronising effects of light. A daily chocolate snack in animals fed ad libitum with chow diet entrains the suprachiasmatic clockwork only under the conditions of constant darkness and decreases the synchronising effects of light. Secondary clocks in the brain outside the suprachiasmatic nuclei are differentially influenced by meal timing. Circadian oscillations can either be highly sensitive to food-related metabolic or reward cues (i.e. their phase is shifted according to the timed meal schedule) in some structures or hardly affected by meal timing (palatable or not) in others. Furthermore, animals will manifest food-anticipatory activity prior to their expected meal time. Anticipation of a palatable or regular meal may rely on a network of brain clocks, involving metabolic and reward systems and the cerebellum.
-
Differentiating between bipolar and unipolar depression in functional and structural MRI studies.
Han, Kyu-Man; De Berardis, Domenico; Fornaro, Michele; Kim, Yong-Ku
2018-03-28
Distinguishing depression in bipolar disorder (BD) from unipolar depression (UD) solely based on clinical clues is difficult, which has led to the exploration of promising neural markers in neuroimaging measures for discriminating between BD depression and UD. In this article, we review structural and functional magnetic resonance imaging (MRI) studies that directly compare UD and BD depression based on neuroimaging modalities including functional MRI studies on regional brain activation or functional connectivity, structural MRI on gray or white matter morphology, and pattern classification analyses using a machine learning approach. Numerous studies have reported distinct functional and structural alterations in emotion- or reward-processing neural circuits between BD depression and UD. Different activation patterns in neural networks including the amygdala, anterior cingulate cortex (ACC), prefrontal cortex (PFC), and striatum during emotion-, reward-, or cognition-related tasks have been reported between BD and UD. A stronger functional connectivity pattern in BD was pronounced in default mode and in frontoparietal networks and brain regions including the PFC, ACC, parietal and temporal regions, and thalamus compared to UD. Gray matter volume differences in the ACC, hippocampus, amygdala, and dorsolateral prefrontal cortex (DLPFC) have been reported between BD and UD, along with a thinner DLPFC in BD compared to UD. BD showed reduced integrity in the anterior part of the corpus callosum and posterior cingulum compared to UD. Several studies performed pattern classification analysis using structural and functional MRI data to distinguish between UD and BD depression using a supervised machine learning approach, which yielded a moderate level of accuracy in classification. Copyright © 2018 Elsevier Inc. All rights reserved.
-
Steiger, Tineke K; Bunzeck, Nico
2017-01-01
Motivation can have invigorating effects on behavior via dopaminergic neuromodulation. While this relationship has mainly been established in theoretical models and studies in younger subjects, the impact of structural declines of the dopaminergic system during healthy aging remains unclear. To investigate this issue, we used electroencephalography (EEG) in healthy young and elderly humans in a reward-learning paradigm. Specifically, scene images were initially encoded by combining them with cues predicting monetary reward (high vs. low reward). Subsequently, recognition memory for the scenes was tested. As a main finding, we can show that response times (RTs) during encoding were faster for high reward predicting images in the young but not elderly participants. This pattern was resembled in power changes in the theta-band (4-7 Hz). Importantly, analyses of structural MRI data revealed that individual reward-related differences in the elderlies' response time could be predicted by the structural integrity of the dopaminergic substantia nigra (SN; as measured by magnetization transfer (MT)). These findings suggest a close relationship between reward-based invigoration, theta oscillations and age-dependent changes of the dopaminergic system.
-
Geographic and temporal differences in band reporting rates for American black ducks
Conroy, M.J.; Blandin, W.W.
1984-01-01
Recoveries of reward- and standard-banded American black ducks (Anas rubripes) were used to estimate band reporting rates and to investigate geographic and temporal variability in reporting rate for 1978-80. Reporting rates were higher close ( 20 km) to the parent banding site in 3 of 42 samples and were higher close to the nearest black duck banding site in 3 of 35 samples., Reporting rates were higher P 0.05) in Canada than in the Atlantic Flyway for the pre-huntIng season 1980-banded sample, but were otherwise similar among regions. Temporal differences over the duration of the study occured, but there were no consistent trends over time. The small number of rejections suggests that there is little evidence of differences in band reporting rate by proximity to banding sites, geographic location, or over time. Thus, the best estimate of band reporting rate for black ducks is a constant 0.43. This estimate should be used with caution, because we believe that it overestimates reporting rate due to the unknown proportion of reward bands found that were not reported.
-
Dopamine controls the neural dynamics of memory signals and retrieval accuracy.
Apitz, Thore; Bunzeck, Nico
2013-11-01
The human brain is capable of differentiating between new and already stored information rapidly to allow optimal behavior and decision-making. Although the neural mechanisms of novelty discrimination were often described as temporally constant (ie, with specific latencies), recent electrophysiological studies have demonstrated that the onset of neural novelty signals (ie, differences in event-related responses to new and old items) can be accelerated by reward motivation. While the precise physiological mechanisms underlying this acceleration remain unclear, the involvement of the neurotransmitter dopamine in both novelty and reward processing suggests that enhanced dopamine levels in the context of reward prospect may have a role. To investigate this hypothesis, we used magnetoencephalography (MEG) in combination with an old/new recognition memory task in which correct discrimination between old and new items was rewarded. Importantly, before the task, human subjects received either 150 mg of the dopamine precursor levodopa or placebo. For the placebo group, old/new signals peaked at ∼100 ms after stimulus onset over left temporal/occipital sensors. In contrast, after levodopa administration earliest old/new effects only emerged after ∼400 ms and retrieval accuracy was reduced as expressed in lower d' values. As such, our results point towards a previously unreported role of dopamine in controlling the chronometry of neural processes underlying the distinction between old and new information. They also suggest that this relationship follows a nonlinear function whereby slightly enhanced dopamine levels accelerate neural/cognitive processes and excessive dopamine levels impair them.
-
Palombo, D J; Keane, M M; Verfaellie, M
2016-08-01
The capacity to envision the future plays an important role in many aspects of cognition, including our ability to make optimal, adaptive choices. Past work has shown that the medial temporal lobe (MTL) is necessary for decisions that draw on episodic future thinking. By contrast, little is known about the role of the MTL in decisions that draw on semantic future thinking. Accordingly, the present study investigated whether the MTL contributes to one form of decision making, namely intertemporal choice, when such decisions depend on semantic consideration of the future. In an intertemporal choice task, participants must select either a smaller amount of money that is available in the present or a larger amount of money that would be available at a future date. Amnesic individuals with MTL damage and healthy control participants performed such a task in which, prior to making a choice, they engaged in a semantic generation exercise, wherein they generated items that they would purchase with the future reward. In experiment 1, we found that, relative to a baseline condition involving standard intertemporal choice, healthy individuals were more inclined to select a larger, later reward over a smaller, present reward after engaging in semantic future thinking. By contrast, amnesic participants were paradoxically less inclined to wait for a future reward following semantic future thinking. This finding suggests that amnesics may have had difficulty "tagging" the generated item(s) as belonging to the future. Critically, experiment 2 showed that when the generated items were presented alongside the intertemporal choices, both controls and amnesic participants shifted to more patient choices. These findings suggest that the MTL is not needed for making optimal decisions that draw on semantic future thinking as long as scaffolding is provided to support accurate time tagging. Together, these findings stand to better clarify the role of the MTL in decision making. Published by Elsevier Ltd.
-
The basolateral amygdala in reward learning and addiction.
Wassum, Kate M; Izquierdo, Alicia
2015-10-01
Sophisticated behavioral paradigms partnered with the emergence of increasingly selective techniques to target the basolateral amygdala (BLA) have resulted in an enhanced understanding of the role of this nucleus in learning and using reward information. Due to the wide variety of behavioral approaches many questions remain on the circumscribed role of BLA in appetitive behavior. In this review, we integrate conclusions of BLA function in reward-related behavior using traditional interference techniques (lesion, pharmacological inactivation) with those using newer methodological approaches in experimental animals that allow in vivo manipulation of cell type-specific populations and neural recordings. Secondly, from a review of appetitive behavioral tasks in rodents and monkeys and recent computational models of reward procurement, we derive evidence for BLA as a neural integrator of reward value, history, and cost parameters. Taken together, BLA codes specific and temporally dynamic outcome representations in a distributed network to orchestrate adaptive responses. We provide evidence that experiences with opiates and psychostimulants alter these outcome representations in BLA, resulting in long-term modified action. Copyright © 2015 Elsevier Ltd. All rights reserved.
-
Discounting of reward sequences: a test of competing formal models of hyperbolic discounting
Zarr, Noah; Alexander, William H.; Brown, Joshua W.
2014-01-01
Humans are known to discount future rewards hyperbolically in time. Nevertheless, a formal recursive model of hyperbolic discounting has been elusive until recently, with the introduction of the hyperbolically discounted temporal difference (HDTD) model. Prior to that, models of learning (especially reinforcement learning) have relied on exponential discounting, which generally provides poorer fits to behavioral data. Recently, it has been shown that hyperbolic discounting can also be approximated by a summed distribution of exponentially discounted values, instantiated in the μAgents model. The HDTD model and the μAgents model differ in one key respect, namely how they treat sequences of rewards. The μAgents model is a particular implementation of a Parallel discounting model, which values sequences based on the summed value of the individual rewards whereas the HDTD model contains a non-linear interaction. To discriminate among these models, we observed how subjects discounted a sequence of three rewards, and then we tested how well each candidate model fit the subject data. The results show that the Parallel model generally provides a better fit to the human data. PMID:24639662
-
Mutual Influence of Reward Anticipation and Emotion on Brain Activity during Memory Retrieval
Yan, Chunping; Liu, Fang; Li, Yunyun; Zhang, Qin; Cui, Lixia
2017-01-01
Previous studies on the joint effect of reward motivation and emotion on memory retrieval have obtained inconsistent results. Furthermore, whether and how any such joint effect might vary over time remains unclear too. Accordingly, using the event-related potential (ERP) measurement of high temporal resolution, our study investigates the cognitive and brain mechanisms of monetary reward and emotion affecting the retrieval processes of episodic memory. Twenty undergraduate and graduate students participated in the research, and our study’s behavioral results indicated that reward (relative to no reward) and negative emotion (relative to positive and neutral emotion) significantly improved recognition performance. The ERP results showed that there were significant interactions between monetary reward and emotion on memory retrieval, and the reward effects of positive, neutral, and negative memory occurred at varied intervals in mean amplitude. The reward effect of positive memory appeared relatively early, at 260–330 ms after the stimulus onset in the frontal-frontocentral area, at 260–500 ms in the centroparietal-parietal area and at 500–700 ms in the frontocentral area. However, the reward effects of neutral and negative memory occurred relatively later, and that of negative memory appeared at 500–700 ms in the frontocentral and centroparietal area and that of neutral memory was at 500–700 ms in the frontocentral and centroparietal-parietal area. Meanwhile, significant FN400 old/new effects were observed in the negative and rewarded positive items, and the old/new effects of negative items appeared earlier at FN400 than positive items. Also, significant late positive component (LPC) old/new effects were found in the positive, negative, and rewarded neutral items. These results suggest that, monetary reward and negative emotion significantly improved recognition performance, and there was a mutual influence between reward and emotion on brain activity during memory retrieval. PMID:29118728
-
Daniel, Reka; Pollmann, Stefan
2010-01-06
The dopaminergic system is known to play a central role in reward-based learning (Schultz, 2006), yet it was also observed to be involved when only cognitive feedback is given (Aron et al., 2004). Within the domain of information-integration category learning, in which information from several stimulus dimensions has to be integrated predecisionally (Ashby and Maddox, 2005), the importance of contingent feedback is well established (Maddox et al., 2003). We examined the common neural correlates of reward anticipation and prediction error in this task. Sixteen subjects performed two parallel information-integration tasks within a single event-related functional magnetic resonance imaging session but received a monetary reward only for one of them. Similar functional areas including basal ganglia structures were activated in both task versions. In contrast, a single structure, the nucleus accumbens, showed higher activation during monetary reward anticipation compared with the anticipation of cognitive feedback in information-integration learning. Additionally, this activation was predicted by measures of intrinsic motivation in the cognitive feedback task and by measures of extrinsic motivation in the rewarded task. Our results indicate that, although all other structures implicated in category learning are not significantly affected by altering the type of reward, the nucleus accumbens responds to the positive incentive properties of an expected reward depending on the specific type of the reward.
-
ERIC Educational Resources Information Center
Hall, Philip S.
2009-01-01
Using rewards to impact students' behavior has long been common practice. However, using reward systems to enhance student learning conveniently masks the larger and admittedly more difficult task of finding and implementing the structure and techniques that children with special needs require to learn. More important, rewarding the child for good…
-
On the asymptotic equivalence between differential Hebbian and temporal difference learning.
Kolodziejski, Christoph; Porr, Bernd; Wörgötter, Florentin
2009-04-01
In this theoretical contribution, we provide mathematical proof that two of the most important classes of network learning-correlation-based differential Hebbian learning and reward-based temporal difference learning-are asymptotically equivalent when timing the learning with a modulatory signal. This opens the opportunity to consistently reformulate most of the abstract reinforcement learning framework from a correlation-based perspective more closely related to the biophysics of neurons.
-
Brain Regions Related to Impulsivity Mediate the Effects of Early Adversity on Antisocial Behavior.
Mackey, Scott; Chaarani, Bader; Kan, Kees-Jan; Spechler, Philip A; Orr, Catherine; Banaschewski, Tobias; Barker, Gareth; Bokde, Arun L W; Bromberg, Uli; Büchel, Christian; Cattrell, Anna; Conrod, Patricia J; Desrivières, Sylvane; Flor, Herta; Frouin, Vincent; Gallinat, Jürgen; Gowland, Penny; Heinz, Andreas; Ittermann, Bernd; Paillère Martinot, Marie-Laure; Artiges, Eric; Nees, Frauke; Papadopoulos-Orfanos, Dimitri; Poustka, Luise; Smolka, Michael N; Jurk, Sarah; Walter, Henrik; Whelan, Robert; Schumann, Gunter; Althoff, Robert R; Garavan, Hugh
2017-08-15
Individual differences in impulsivity and early adversity are known to be strong predictors of adolescent antisocial behavior. However, the neurobiological bases of impulsivity and their relation to antisocial behavior and adversity are poorly understood. Impulsivity was estimated with a temporal discounting task. Voxel-based morphometry was used to determine the brain structural correlates of temporal discounting in a large cohort (n = 1830) of 14- to 15-year-old children. Mediation analysis was then used to determine whether the volumes of brain regions associated with temporal discounting mediate the relation between adverse life events (e.g., family conflict, serious accidents) and antisocial behaviors (e.g., precocious sexual activity, bullying, illicit substance use). Greater temporal discounting (more impulsivity) was associated with 1) lower volume in frontomedial cortex and bilateral insula and 2) greater volume in a subcortical region encompassing the ventral striatum, hypothalamus and anterior thalamus. The volume ratio between these cortical and subcortical regions was found to partially mediate the relation between adverse life events and antisocial behavior. Temporal discounting is related to regions of the brain involved in reward processing and interoception. The results support a developmental imbalance model of impulsivity and are consistent with the idea that negative environmental factors can alter the developing brain in ways that promote antisocial behavior. Copyright © 2016 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.
-
Reward processing in neurodegenerative disease
Perry, David C.; Kramer, Joel H.
2015-01-01
Representation of reward value involves a distributed network including cortical and subcortical structures. Because neurodegenerative illnesses target specific anatomic networks that partially overlap with the reward circuit they would be predicted to have distinct impairments in reward processing. This review presents the existing evidence of reward processing changes in neurodegenerative diseases including mild cognitive impairment, Alzheimer's disease, frontotemporal dementia, amyotrophic lateral sclerosis, Parkinson's disease, and Huntington's disease, as well as in healthy aging. Carefully distinguishing the different aspects of reward processing (primary rewards, secondary rewards, reward-based learning, and reward-based decision-making) and using tasks that differentiate the stages of processing reward will lead to improved understanding of this fundamental process and clarify a contributing cause of behavioral change in these illnesses. PMID:24417286
-
Phillips, Mary L; Swartz, Holly A.
2014-01-01
Objective This critical review appraises neuroimaging findings in bipolar disorder in emotion processing, emotion regulation, and reward processing neural circuitry, to synthesize current knowledge of the neural underpinnings of bipolar disorder, and provide a neuroimaging research “roadmap” for future studies. Method We examined findings from all major studies in bipolar disorder that used fMRI, volumetric analyses, diffusion imaging, and resting state techniques, to inform current conceptual models of larger-scale neural circuitry abnormalities in bipolar disorder Results Bipolar disorder can be conceptualized in neural circuitry terms as parallel dysfunction in bilateral prefrontal cortical (especially ventrolateral prefrontal cortical)-hippocampal-amygdala emotion processing and emotion regulation neural circuitries, together with an “overactive” left-sided ventral striatal-ventrolateral and orbitofrontal cortical reward processing circuitry, that result in characteristic behavioral abnormalities associated with bipolar disorder: emotional lability, emotional dysregulation and heightened reward sensitivity. A potential structural basis for these functional abnormalities are gray matter decreases in prefrontal and temporal cortices, amygdala and hippocampus, and fractional anisotropy decreases in white matter tracts connecting prefrontal and subcortical regions. Conclusion Neuroimaging studies of bipolar disorder clearly demonstrate abnormalities in neural circuitries supporting emotion processing, emotion regulation and reward processing, although there are several limitations to these studies. Future neuroimaging research in bipolar disorder should include studies adopting dimensional approaches; larger studies examining neurodevelopmental trajectories in bipolar disorder and at-risk youth; multimodal neuroimaging studies using integrated systems approaches; and studies using pattern recognition approaches to provide clinically useful, individual-level data. Such studies will help identify clinically-relevant biomarkers to guide diagnosis and treatment decision-making for individuals with bipolar disorder. PMID:24626773
-
Seo, Hyojung; Lee, Daeyeol
2008-01-01
The process of decision making in humans and other animals is adaptive and can be tuned through experience so as to optimize the outcomes of their choices in a dynamic environment. Previous studies have demonstrated that the anterior cingulate cortex plays an important role in updating the animal’s behavioral strategies when the action-outcome contingencies change. Moreover, neurons in the anterior cingulate cortex often encode the signals related to expected or actual reward. We investigated whether reward-related activity in the anterior cingulate cortex is affected by the animal’s previous reward history. This was tested in rhesus monkeys trained to make binary choices in a computer-simulated competitive zero-sum game. The animal’s choice behavior was relatively close to the optimal strategy, but also revealed small but systematic biases that are consistent with the use of a reinforcement learning algorithm. In addition, the activity of neurons in the dorsal anterior cingulate cortex that was related to the reward received by the animal in a given trial was often modulated by the rewards in the previous trials. Some of these neurons encoded the rate of rewards in previous trials, whereas others displayed activity modulations more closely related to the reward prediction errors. By contrast, signals related to the animal’s choices were only weakly represented in this cortical area. These results suggest that neurons in the dorsal anterior cingulate cortex might be involved in the subjective evaluation of choice outcomes based on the animal’s reward history. PMID:17670983
-
Striatal dopamine transmission in healthy humans during a passive monetary reward task.
Hakyemez, Hélène S; Dagher, Alain; Smith, Stephen D; Zald, David H
2008-02-15
Research on dopamine (DA) transmission has emphasized the importance of increased phasic DA cell firing in the presence of unpredictable rewards. Using [(11)C]raclopride PET, we previously reported that DA transmission was both suppressed and enhanced in different regions of the striatum during an unpredictable reward task [Zald, D.H., Boileau, I., El Dearedy, W., Gunn, R., McGlone, F., Dichter, G.S. et al. (2004). Dopamine transmission in the human striatum during monetary reward tasks. J. Neurosci. 24, 4105-4112]. However, it was unclear if reductions in DA release during this task reflected a response to the high proportion of nonrewarding trials, and whether the behavioral demands of the task influenced the observed response. To test these issues, we presented 10 healthy subjects with an automated (passive) roulette wheel game in which the amount of reward and its timing were unpredictable and the rewarding trials greatly outnumbered the nonrewarding ones. As in the previous study, DA transmission in the putamen was significantly suppressed relative to a predictable control condition. A similar suppression occurred when subjects were presented with temporally unpredictable novel pictures and sounds. At present, models of DA functioning during reward do not account for this suppression, but given that it has been observed in two different studies using different reward paradigms, this phenomenon warrants attention. Neither the unpredictable reward nor the novelty conditions produced consistent increases in striatal DA transmission. These data suggest that active behavioral engagement may be necessary to observe robust statewise increases in DA release in the striatum.
-
Neural reactivity to reward in school-age offspring of depressed mothers.
Wiggins, Jillian Lee; Schwartz, Karen T G; Kryza-Lacombe, Maria; Spechler, Philip A; Blankenship, Sarah L; Dougherty, Lea R
2017-05-01
Identifying neural profiles predictive of future psychopathology in at-risk individuals is important to efficiently direct preventive care. Alterations in reward processing may be a risk factor for depression. The current study characterized neural substrates of reward processing in children at low- and high-risk for psychopathology due to maternal depression status. Children with (n=27) and without (n=19) maternal depression (ages 5.9-9.6 years) performed a monetary incentive delay task in which they received rewards, if they successfully hit a target, or no reward regardless of performance, during fMRI acquisition. Multiple dorsal prefrontal, temporal, and striatal regions showed significant Group (high- vs. low-risk)×Performance (hit vs. miss)×Condition (no reward vs. reward) interactions in a whole-brain analysis. All regions exhibited similar patterns, whereby the high-risk group showed blunted activation differences between trials with vs. without rewards when participants hit the target. Moreover, high-risk children showed activation differences between trials with vs. without rewards in the opposite direction, compared to the low-risk group, when they missed the target. This study had a modest sample size, though larger than existing studies. Children with maternal depression are at elevated risk for future psychopathology, yet not all experience clinically significant symptoms; longitudinal research is necessary to fully track the pathway from risk to disorder. Children of depressed mothers exhibited attenuated neural activation differences and activation patterns opposite to children without depressed mothers. Our findings may provide targets for hypothesis-driven preventive interventions and lead to earlier identification of individuals at risk. Copyright © 2017 Elsevier B.V. All rights reserved.
-
Noori, Hamid R; Cosa Linan, Alejandro; Spanagel, Rainer
2016-09-01
Cue reactivity to natural and social rewards is essential for motivational behavior. However, cue reactivity to drug rewards can also elicit craving in addicted subjects. The degree to which drug and natural rewards share neural substrates is not known. The objective of this study is to conduct a comprehensive meta-analysis of neuroimaging studies on drug, gambling and natural stimuli (food and sex) to identify the common and distinct neural substrates of cue reactivity to drug and natural rewards. Neural cue reactivity studies were selected for the meta-analysis by means of activation likelihood estimations, followed by sensitivity and clustering analyses of averaged neuronal response patterns. Data from 176 studies (5573 individuals) suggests largely overlapping neural response patterns towards all tested reward modalities. Common cue reactivity to natural and drug rewards was expressed by bilateral neural responses within anterior cingulate gyrus, insula, caudate head, inferior frontal gyrus, middle frontal gyrus and cerebellum. However, drug cues also generated distinct activation patterns in medial frontal gyrus, middle temporal gyrus, posterior cingulate gyrus, caudate body and putamen. Natural (sexual) reward cues induced unique activation of the pulvinar in thalamus. Neural substrates of cue reactivity to alcohol, drugs of abuse, food, sex and gambling are largely overlapping and comprise a network that processes reward, emotional responses and habit formation. This suggests that cue-mediated craving involves mechanisms that are not exclusive for addictive disorders but rather resemble the intersection of information pathways for processing reward, emotional responses, non-declarative memory and obsessive-compulsive behavior. Copyright © 2016 Elsevier B.V. and ECNP. All rights reserved.
-
Personality influences temporal discounting preferences: behavioral and brain evidence.
Manning, Joshua; Hedden, Trey; Wickens, Nina; Whitfield-Gabrieli, Susan; Prelec, Drazen; Gabrieli, John D E
2014-09-01
Personality traits are stable predictors of many life outcomes that are associated with important decisions that involve tradeoffs over time. Therefore, a fundamental question is how tradeoffs over time vary from person to person in relation to stable personality traits. We investigated the influence of personality, as measured by the Five-Factor Model, on time preferences and on neural activity engaged by intertemporal choice. During functional magnetic resonance imaging (fMRI), participants made choices between smaller-sooner and larger-later monetary rewards. For each participant, we estimated a constant-sensitivity discount function that dissociates impatience (devaluation of future consequences) from time sensitivity (consistency with rational, exponential discounting). Overall, higher neuroticism was associated with a relatively greater preference for immediate rewards and higher conscientiousness with a relatively greater preference for delayed rewards. Specifically, higher conscientiousness correlated positively with lower short-term impatience and more exponential time preferences, whereas higher neuroticism (lower emotional stability) correlated positively with higher short-term impatience and less exponential time preferences. Cognitive-control and reward brain regions were more activated when higher conscientiousness participants selected a smaller-sooner reward and, conversely, when higher neuroticism participants selected a larger-later reward. The greater activations that occurred when choosing rewards that contradicted personality predispositions may reflect the greater recruitment of mental resources needed to override those predispositions. These findings reveal that stable personality traits fundamentally influence how rewards are chosen over time. Copyright © 2014 Elsevier Inc. All rights reserved.
-
Duprat, Romain; De Raedt, Rudi; Wu, Guo-Rong; Baeken, Chris
2016-01-01
Repetitive transcranial magnetic stimulation over the left dorsolateral prefrontal cortex (DLPFC) has been documented to influence striatal and orbitofrontal dopaminergic activity implicated in reward processing. However, the exact neuropsychological mechanisms of how DLPFC stimulation may affect the reward system and how trait hedonic capacity may interact with the effects remains to be elucidated. In this sham-controlled study in healthy individuals, we investigated the effects of a single session of neuronavigated intermittent theta burst stimulation (iTBS) on reward responsiveness, as well as the influence of trait hedonic capacity. We used a randomized crossover single session iTBS design with an interval of 1 week. We assessed reward responsiveness using a rewarded probabilistic learning task and measured individual trait hedonic capacity (the ability to experience pleasure) with the temporal experience of pleasure scale questionnaire. As expected, the participants developed a response bias toward the most rewarded stimulus (rich stimulus). Reaction time and accuracy for the rich stimulus were respectively shorter and higher as compared to the less rewarded stimulus (lean stimulus). Active or sham stimulation did not seem to influence the outcome. However, when taking into account individual trait hedonic capacity, we found an early significant increase in the response bias only after active iTBS. The higher the individual's trait hedonic capacity, the more the response bias toward the rich stimulus increased after the active stimulation. When taking into account trait hedonic capacity, one active iTBS session over the left DLPFC improved reward responsiveness in healthy male participants with higher hedonic capacity. This suggests that individual differences in hedonic capacity may influence the effects of iTBS on the reward system.
-
Reinforcement Learning Using a Continuous Time Actor-Critic Framework with Spiking Neurons
Frémaux, Nicolas; Sprekeler, Henning; Gerstner, Wulfram
2013-01-01
Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity. PMID:23592970
-
Reinforcement learning using a continuous time actor-critic framework with spiking neurons.
Frémaux, Nicolas; Sprekeler, Henning; Gerstner, Wulfram
2013-04-01
Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.
-
Völter, Christoph J; Call, Josep
2014-05-01
Whether nonhuman primates understand causal relations beyond mere associations is still a matter of debate. We presented all four species of nonhuman great apes (N = 36) with a choice between 2 opaque, upside down cups after displacing them out of sight from their starting positions. Crucially, 1 of them had left a yogurt trail behind it. Great apes spontaneously used the trail to select the yogurt baited cup. Follow-up experiments demonstrated that chimpanzees distinguished trails based on the temporal order of cause and effect by ignoring trails that were already present before the reward was hidden. Additionally, chimpanzees did not select cups based on the amount of yogurt near them but instead preferred cups that signaled the endpoint of the trail. We conclude that apes' choices reveal sensitivity to a causal relation between cause (reward) and effect (trail) including their temporal order. ©2014 APA, all rights reserved.
-
Steele, Catherine C.; Peterson, Jennifer R.; Marshall, Andrew T.; Stuebing, Sarah L.; Kirkpatrick, Kimberly
2017-01-01
The nucleus accumbens core (NAc) has long been recognized as an important contributor to the computation of reward value that is critical for impulsive choice behavior. Impulsive choice refers to choosing a smaller-sooner (SS) over a larger-later (LL) reward when the LL is more optimal in terms of the rate of reward delivery. Two experiments examined the role of the NAc in impulsive choice and its component processes of delay and magnitude processing. Experiment 1 delivered an impulsive choice task with manipulations of LL reward magnitude, followed by a reward magnitude discrimination task. Experiment 2 tested impulsive choice under manipulations of LL delay, followed by temporal bisection and progressive interval tasks. NAc lesions, in comparison to sham control lesions, produced suboptimal preferences that resulted in lower reward earning rates, and led to reduced sensitivity to magnitude and delay within the impulsive choice task. The secondary tasks revealed intact reward magnitude and delay discrimination abilities, but the lesion rats persisted in responding more as the progressive interval increased during the session. The results suggest that the NAc is most critical for demonstrating good sensitivity to magnitude and delay, and adjusting behavior accordingly. Ultimately, the NAc lesions induced suboptimal choice behavior rather than simply promoting impulsive choice, suggesting that an intact NAc is necessary for optimal decision making. PMID:29146281
-
ERIC Educational Resources Information Center
What Works Clearinghouse, 2010
2010-01-01
This paper presents an updated WWC (What Works Clearinghouse) Review of the Article "Culture and the Interaction of Student Ethnicity with Reward Structure in Group Learning". The study examined the effects of different reward systems used in group learning situations on the math skills of African-American and White students. The…
-
ERIC Educational Resources Information Center
What Works Clearinghouse, 2010
2010-01-01
"Culture and the Interaction of Student Ethnicity with Reward Structure in Group Learning" examined the effects of different reward systems used in group learning situations on the math skills of African-American and white students. The study analyzed data on 75 African-American and 57 white fourth- and fifth-grade students from urban…
-
Webb, Ian C; Baltazar, Ricardo M; Lehman, Michael N; Coolen, Lique M
2009-11-01
Reward is mediated by a distributed series of midbrain and basal forebrain structures collectively referred to as the brain reward system. Recent evidence indicates that an additional regulatory system, the circadian system, can modulate reward-related learning. Diurnal or circadian changes in drug self-administration, responsiveness to drugs of abuse and reward to natural stimuli have been reported. These variations are associated with daily rhythms in mesolimbic electrical activity, dopamine synthesis and metabolism, and local clock gene oscillations. Conversely, the presentation of rewards appears capable of influencing circadian timing. Rodents can anticipate a daily mealtime by the entrainment of a series of oscillators that are anatomically distinct from the suprachiasmatic nucleus. Other work has indicated that restricted access to non-nutritive reinforcers (e.g. drugs of abuse, sex) or to palatable food in the absence of an energy deficit is capable of inducing relatively weak anticipatory activity, suggesting that reward alone is sufficient to induce anticipation. Recent attempts to elucidate the neural correlates of anticipation have revealed that both restricted feeding and restricted palatable food access can entrain clock gene expression in many reward-related corticolimbic structures. By contrast, restricted feeding alone can induce or entrain clock gene expression in hypothalamic nuclei involved in energy homeostasis. Thus, under ad libitum feeding conditions, the weak anticipatory activity induced by restricted reward presentation may result from the entrainment of reward-associated corticolimbic structures. The additional induction or entrainment of oscillators in hypothalamic regulatory areas may contribute to the more robust anticipatory activity associated with restricted feeding schedules.
-
Tosun, Tuğçe; Gür, Ezgi; Balcı, Fuat
2016-01-01
Animals can shape their timed behaviors based on experienced probabilistic relations in a nearly optimal fashion. On the other hand, it is not clear if they adopt these timed decisions by making computations based on previously learnt task parameters (time intervals, locations, and probabilities) or if they gradually develop their decisions based on trial and error. To address this question, we tested mice in the timed-switching task, which required them to anticipate when (after a short or long delay) and at which of the two delay locations a reward would be presented. The probability of short trials differed between test groups in two experiments. Critically, we first trained mice on relevant task parameters by signaling the active trial with a discriminative stimulus and delivered the corresponding reward after the associated delay without any response requirement (without inducing switching behavior). During the test phase, both options were presented simultaneously to characterize the emergence and temporal characteristics of the switching behavior. Mice exhibited timed-switching behavior starting from the first few test trials, and their performance remained stable throughout testing in the majority of the conditions. Furthermore, as the probability of the short trial increased, mice waited longer before switching from the short to long location (experiment 1). These behavioral adjustments were in directions predicted by reward maximization. These results suggest that rather than gradually adjusting their time-dependent choice behavior, mice abruptly adopted temporal decision strategies by directly integrating their previous knowledge of task parameters into their timed behavior, supporting the model-based representational account of temporal risk assessment. PMID:26733674
-
Quicker Q-Learning in Multi-Agent Systems
NASA Technical Reports Server (NTRS)
Agogino, Adrian K.; Tumer, Kagan
2005-01-01
Multi-agent learning in Markov Decisions Problems is challenging because of the presence ot two credit assignment problems: 1) How to credit an action taken at time step t for rewards received at t' greater than t; and 2) How to credit an action taken by agent i considering the system reward is a function of the actions of all the agents. The first credit assignment problem is typically addressed with temporal difference methods such as Q-learning OK TD(lambda) The second credit assi,onment problem is typically addressed either by hand-crafting reward functions that assign proper credit to an agent, or by making certain independence assumptions about an agent's state-space and reward function. To address both credit assignment problems simultaneously, we propose the Q Updates with Immediate Counterfactual Rewards-learning (QUICR-learning) designed to improve both the convergence properties and performance of Q-learning in large multi-agent problems. Instead of assuming that an agent s value function can be made independent of other agents, this method suppresses the impact of other agents using counterfactual rewards. Results on multi-agent grid-world problems over multiple topologies show that QUICR-learning can achieve up to thirty fold improvements in performance over both conventional and local Q-learning in the largest tested systems.
-
Goodman, Ronald N; Rietschel, Jeremy C; Roy, Anindo; Jung, Brian C; Diaz, Jason; Macko, Richard F; Forrester, Larry W
2014-01-01
Robotics is rapidly emerging as a viable approach to enhance motor recovery after disabling stroke. Current principles of cognitive motor learning recognize a positive relationship between reward and motor learning. Yet no prior studies have established explicitly whether reward improves the rate or efficacy of robotics-assisted rehabilitation or produces neurophysiologic adaptations associated with motor learning. We conducted a 3 wk, 9-session clinical pilot with 10 people with chronic hemiparetic stroke, randomly assigned to train with an impedance-controlled ankle robot (anklebot) under either high reward (HR) or low reward conditions. The 1 h training sessions entailed playing a seated video game by moving the paretic ankle to hit moving onscreen targets with the anklebot only providing assistance as needed. Assessments included paretic ankle motor control, learning curves, electroencephalograpy (EEG) coherence and spectral power during unassisted trials, and gait function. While both groups exhibited changes in EEG, the HR group had faster learning curves (p = 0.05), smoother movements (p
-
Gray, Joshua C; Amlung, Michael T; Palmer, Abraham A; MacKillop, James
2016-09-01
The 27-item Monetary Choice Questionnaire (MCQ; Kirby, Petry, & Bickel, 1999) and 30-item Probability Discounting Questionnaire (PDQ; Madden, Petry, & Johnson, 2009) are widely used, validated measures of preferences for immediate versus delayed rewards and guaranteed versus risky rewards, respectively. The MCQ measures delayed discounting by asking individuals to choose between rewards available immediately and larger rewards available after a delay. The PDQ measures probability discounting by asking individuals to choose between guaranteed rewards and a chance at winning larger rewards. Numerous studies have implicated these measures in addiction and other health behaviors. Unlike typical self-report measures, the MCQ and PDQ generate inferred hyperbolic temporal and probability discounting functions by comparing choice preferences to arrays of functions to which the individual items are preconfigured. This article provides R and SPSS syntax for processing the MCQ and PDQ. Specifically, for the MCQ, the syntax generates k values, consistency of the inferred k, and immediate choice ratios; for the PDQ, the syntax generates h indices, consistency of the inferred h, and risky choice ratios. The syntax is intended to increase the accessibility of these measures, expedite the data processing, and reduce risk for error. © 2016 Society for the Experimental Analysis of Behavior.
-
QUICR-learning for Multi-Agent Coordination
NASA Technical Reports Server (NTRS)
Agogino, Adrian K.; Tumer, Kagan
2006-01-01
Coordinating multiple agents that need to perform a sequence of actions to maximize a system level reward requires solving two distinct credit assignment problems. First, credit must be assigned for an action taken at time step t that results in a reward at time step t > t. Second, credit must be assigned for the contribution of agent i to the overall system performance. The first credit assignment problem is typically addressed with temporal difference methods such as Q-learning. The second credit assignment problem is typically addressed by creating custom reward functions. To address both credit assignment problems simultaneously, we propose the "Q Updates with Immediate Counterfactual Rewards-learning" (QUICR-learning) designed to improve both the convergence properties and performance of Q-learning in large multi-agent problems. QUICR-learning is based on previous work on single-time-step counterfactual rewards described by the collectives framework. Results on a traffic congestion problem shows that QUICR-learning is significantly better than a Q-learner using collectives-based (single-time-step counterfactual) rewards. In addition QUICR-learning provides significant gains over conventional and local Q-learning. Additional results on a multi-agent grid-world problem show that the improvements due to QUICR-learning are not domain specific and can provide up to a ten fold increase in performance over existing methods.
-
Brzosko, Zuzanna; Zannone, Sara; Schultz, Wolfram
2017-01-01
Spike timing-dependent plasticity (STDP) is under neuromodulatory control, which is correlated with distinct behavioral states. Previously, we reported that dopamine, a reward signal, broadens the time window for synaptic potentiation and modulates the outcome of hippocampal STDP even when applied after the plasticity induction protocol (Brzosko et al., 2015). Here, we demonstrate that sequential neuromodulation of STDP by acetylcholine and dopamine offers an efficacious model of reward-based navigation. Specifically, our experimental data in mouse hippocampal slices show that acetylcholine biases STDP toward synaptic depression, whilst subsequent application of dopamine converts this depression into potentiation. Incorporating this bidirectional neuromodulation-enabled correlational synaptic learning rule into a computational model yields effective navigation toward changing reward locations, as in natural foraging behavior. Thus, temporally sequenced neuromodulation of STDP enables associations to be made between actions and outcomes and also provides a possible mechanism for aligning the time scales of cellular and behavioral learning. DOI: http://dx.doi.org/10.7554/eLife.27756.001 PMID:28691903
-
Multi Agent Reward Analysis for Learning in Noisy Domains
NASA Technical Reports Server (NTRS)
Tumer, Kagan; Agogino, Adrian K.
2005-01-01
In many multi agent learning problems, it is difficult to determine, a priori, the agent reward structure that will lead to good performance. This problem is particularly pronounced in continuous, noisy domains ill-suited to simple table backup schemes commonly used in TD(lambda)/Q-learning. In this paper, we present a new reward evaluation method that allows the tradeoff between coordination among the agents and the difficulty of the learning problem each agent faces to be visualized. This method is independent of the learning algorithm and is only a function of the problem domain and the agents reward structure. We then use this reward efficiency visualization method to determine an effective reward without performing extensive simulations. We test this method in both a static and a dynamic multi-rover learning domain where the agents have continuous state spaces and where their actions are noisy (e.g., the agents movement decisions are not always carried out properly). Our results show that in the more difficult dynamic domain, the reward efficiency visualization method provides a two order of magnitude speedup in selecting a good reward. Most importantly it allows one to quickly create and verify rewards tailored to the observational limitations of the domain.
-
The Nucleus Accumbens and Pavlovian Reward Learning
Day, Jeremy J.
2011-01-01
The ability to form associations between predictive environmental events and rewarding outcomes is a fundamental aspect of learned behavior. This apparently simple ability likely requires complex neural processing evolved to identify, seek, and utilize natural rewards and redirect these activities based on updated sensory information. Emerging evidence from both animal and human research suggests that this type of processing is mediated in part by the nucleus accumbens and a closely associated network of brain structures. The nucleus accumbens is required for a number of reward-related behaviors, and processes specific information about reward availability, value, and context. Additionally, this structure is critical for the acquisition and expression of most Pavlovian stimulus-reward relationships, and cues that predict rewards produce robust changes in neural activity in the nucleus accumbens. While processing within the nucleus accumbens may enable or promote Pavlovian reward learning in natural situations, it has also been implicated in aspects of human drug addiction, including the ability of drug-paired cues to control behavior. This article will provide a critical review of the existing animal and human literature concerning the role of the NAc in Pavlovian learning with non-drug rewards and consider some clinical implications of these findings. PMID:17404375
-
The Tenure of Private College and University Presidents
ERIC Educational Resources Information Center
Langbert, Mitchell
2012-01-01
This study fills several gaps. Most turnaround studies ignore post-turnaround executive rewards, and most studies of executive rewards ignore both the effects on rewards of achieving a turnaround and length of service, or tenure, as an element of the reward structure. Previous research about the length of college presidents' tenure in office has…
-
Urošević, Snežana; Collins, Paul; Muetzel, Ryan; Lim, Kelvin O; Luciana, Monica
2014-08-01
Adolescence is characterized by complex developmental processes that impact behavior, biology, and social functioning. Two such adolescence-specific processes are puberty and increases in reward sensitivity. Relations between these processes are poorly understood. The present study focused on examining unique effects of puberty, age, and sex on reward and threat sensitivities and volumes of subcortical brain structures relevant for reward/threat processing in a healthy sample of 9-18year-olds. Unlike age, pubertal status had a significant unique positive relationship with reward sensitivity. In addition, there was a trend for adolescent females to exhibit higher threat sensitivity with more advanced pubertal development and higher reward and threat sensitivity with older age. Similarly, there were significant puberty by sex interaction effects on striatal volumes, i.e., left nucleus accumbens and right pallidum. The present pattern of results suggests that pubertal development, independent of chronological age, is uniquely associated with reward hypersensitivity and with structural differences in striatal regions implicated in reward processing. Copyright © 2014 Elsevier Inc. All rights reserved.
-
How to cheat when you cannot lie? Deceit pollination in Begonia gracilis.
Castillo, Reyna A; Caballero, Helga; Boege, Karina; Fornoni, Juan; Domínguez, César A
2012-07-01
Mimicry between rewarding and non-rewarding flowers within individuals has been accepted as a strategy favored by selection to deceive pollinators. It has been proposed that this mechanism relies on the exploitation of pollinator's sensory biases, but field evidence is still scarce. In this study, we describe the mechanism of deceit pollination in the monoecious herb Begonia gracilis, a species with exposed rewarding structures (pollen) and intersexual mimicry. Specifically, we test the role of mimicry and exploitation of sensory biases on the reproductive success of male (pollination visitation) and female flowers (probability of setting fruits). We show that pollinators' perception of the amount of reward provided by male flowers is influenced by the independent variation in the sizes of the androecium and the perianth. Large rewarding structures and small perianths were preferred by pollinators, suggesting a central role of the relative size of the rewarding structure on pollinators' foraging decisions. Hence, rewarding male flowers cheat pollinators by exploiting their sensory biases, a strategy followed by non-rewarding female flowers. We suggest that intersexual mimicry operates through the functional resemblance of male flowers' deceit strategy. Artificial manipulation of the flowers supports our findings in natural conditions. Overall, we propose that the continuous and independent variation in the size of the perianth and the reproductive organs among male and female flowers could itself be adaptive.
-
Schneider, S; Brassen, S; Bromberg, U; Banaschewski, T; Conrod, P; Flor, H; Gallinat, J; Garavan, Hugh; Heinz, A; Martinot, J-L; Nees, F; Rietschel, M; Smolka, M N; Ströhle, A; Struve, M; Schumann, G; Büchel, C
2012-01-01
Considerable animal and human research has been dedicated to the effects of parenting on structural brain development, focusing on hippocampal and prefrontal areas. Conversely, although functional imaging studies suggest that the neural reward circuitry is involved in parental affection, little is known about mothers' interpersonal qualities in relation to their children's brain structure and function. Moreover, gender differences concerning the effect of maternal qualities have rarely been investigated systematically. In 63 adolescents, we assessed structural and functional magnetic resonance imaging as well as interpersonal affiliation in their mothers. This allowed us to associate maternal affiliation with gray matter density and neural responses during different phases of the well-established Monetary Incentive Delay task. Maternal affiliation was positively associated with hippocampal and orbitofrontal gray matter density. Moreover, in the feedback of reward hit as compared with reward miss, an association with caudate activation was found. Although no significant gender effects were observed in these associations, during reward feedback as compared with baseline, maternal affiliation was significantly associated with ventral striatal and caudate activation only in females. Our findings demonstrate that maternal interpersonal affiliation is related to alterations in both the brain structure and reward-related activation in healthy adolescents. Importantly, the pattern is in line with typical findings in depression and post-traumatic stress disorder, suggesting that a lack of maternal affiliation might have a role in the genesis of mental disorders. PMID:23149446
-
Utilization of reward-prospect enhances preparatory attention and reduces stimulus conflict.
van den Berg, Berry; Krebs, Ruth M; Lorist, Monicque M; Woldorff, Marty G
2014-06-01
The prospect of gaining money is an incentive widely at play in the real world. Such monetary motivation might have particularly strong influence when the cognitive system is challenged, such as when needing to process conflicting stimulus inputs. Here, we employed manipulations of reward-prospect and attentional-preparation levels in a cued-Stroop stimulus conflict task, along with the high temporal resolution of electrical brain recordings, to provide insight into the mechanisms by which reward-prospect and attention interact and modulate cognitive task performance. In this task, the cue indicated whether or not the participant needed to prepare for an upcoming Stroop stimulus and, if so, whether there was the potential for monetary reward (dependent on performance on that trial). Both cued attention and cued reward-prospect enhanced preparatory neural activity, as reflected by increases in the hallmark attention-related negative-polarity ERP slow wave (contingent negative variation [CNV]) and reductions in oscillatory Alpha activity, which was followed by enhanced processing of the subsequent Stroop stimulus. In addition, similar modulations of preparatory neural activity (larger CNVs and reduced Alpha) predicted shorter versus longer response times (RTs) to the subsequent target stimulus, consistent with such modulations reflecting trial-to-trial variations in attention. Particularly striking were the individual differences in the utilization of reward-prospect information. In particular, the size of the reward effects on the preparatory neural activity correlated across participants with the degree to which reward-prospect both facilitated overall task performance (shorter RTs) and reduced conflict-related behavioral interference. Thus, the prospect of reward appears to recruit attentional preparation circuits to enhance processing of task-relevant target information.
-
Utilization of reward-prospect enhances preparatory attention and reduces stimulus conflict
van den Berg, Berry; Krebs, Ruth M.; Lorist, Monicque M.; Woldorff, Marty G.
2015-01-01
The prospect of gaining money is an incentive widely at play in the real world. Such monetary motivation might have particularly strong influence when the cognitive system is challenged, such as when needing to process conflicting stimulus inputs. Here, we employed manipulations of reward-prospect and attentional-preparation levels in a cued-Stroop stimulus-conflict task, along with the high temporal resolution of electrical brain recordings, to provide insight into the mechanisms by which reward-prospect and attention interact and modulate cognitive-task performance. In this task the cue indicated whether or not the subject needed to prepare for an upcoming Stroop stimulus, and if so, whether there was the potential for monetary reward (dependent on performance on that trial). Both cued-attention and cued-reward-prospect enhanced preparatory neural activity, as reflected by increases in the hallmark attention-related negative-polarity ERP slow wave (CNV) and reductions in oscillatory Alpha activity, which was followed by enhanced processing of the subsequent Stroop stimulus. In addition, similar modulations of preparatory neural activity (larger CNVs and reduced Alpha) predicted faster versus slower response times (RTs) to the subsequent target stimulus, consistent with such modulations reflecting trial-to-trial variations in attention. Particularly striking were the individual differences in the utilization of reward-prospect information. In particular, the size of the reward effects on the preparatory neural activity correlated across-subjects with the degree to which reward-prospect both facilitated overall task performance (faster RTs) and reduced conflict-related behavioral interference. Thus, the prospect of reward appears to recruit attentional preparation circuits to enhance processing of task-relevant target information. PMID:24820263
-
NASA Astrophysics Data System (ADS)
Hadzibeganovic, Tarik; Stauffer, Dietrich; Han, Xiao-Pu
2018-04-01
Cooperation is fundamental for the long-term survival of biological, social, and technological networks. Previously, mechanisms for the enhancement of cooperation, such as network reciprocity, have largely been studied in isolation and with often inconclusive findings. Here, we present an evolutionary, multiagent-based, and spatially explicit computer model to specifically address the interactive interplay between such mechanisms. We systematically investigate the effects of phenotypic diversity, network structure, and rewards on cooperative behavior emerging in a population of reproducing artificial decision makers playing tag-mediated evolutionary games. Cooperative interactions are rewarded such that both the benefits of recipients and costs of donators are affected by the reward size. The reward size is determined by the number of cooperative acts occurring within a given reward time frame. Our computational experiments reveal that small reward frames promote unconditional cooperation in populations with both low and high diversity, whereas large reward frames lead to cycles of conditional and unconditional strategies at high but not at low diversity. Moreover, an interaction between rewards and spatial structure shows that relative to small reward frames, there is a strong difference between the frequency of conditional cooperators populating rewired versus non-rewired networks when the reward frame is large. Notably, in a less diverse population, the total number of defections is comparable across different network topologies, whereas in more diverse environments defections become more frequent in a regularly structured than in a rewired, small-world network of contacts. Acknowledging the importance of such interaction effects in social dilemmas will have inevitable consequences for the future design of cooperation-enhancing protocols in large-scale, distributed, and decentralized systems such as peer-to-peer networks.
-
MacKillop, James; Amlung, Michael T; Wier, Lauren M; David, Sean P; Ray, Lara A; Bickel, Warren K; Sweet, Lawrence H
2012-04-30
Neuroeconomics integrates behavioral economics and cognitive neuroscience to understand the neurobiological basis for normative and maladaptive decision making. Delay discounting is a behavioral economic index of impulsivity that reflects capacity to delay gratification and has been consistently associated with nicotine dependence. This preliminary study used functional magnetic resonance imaging to examine delay discounting for money and cigarette rewards in 13 nicotine dependent adults. Significant differences between preferences for smaller immediate rewards and larger delayed rewards were evident in a number of regions of interest (ROIs), including the medial prefrontal cortex, anterior insular cortex, middle temporal gyrus, middle frontal gyrus, and cingulate gyrus. Significant differences between money and cigarette rewards were generally lateralized, with cigarette choices associated with left hemisphere activation and money choices associated with right hemisphere activation. Specific ROI differences included the posterior parietal cortex, medial and middle frontal gyrus, ventral striatum, temporoparietal cortex, and angular gyrus. Impulsivity as measured by behavioral choices was significantly associated with both individual ROIs and a combined ROI model. These findings provide initial evidence in support of applying a neuroeconomic approach to understanding nicotine dependence. Copyright © 2012 Elsevier Ireland Ltd. All rights reserved.
-
Williamson, Rebecca A.; Latzman, Robert D.; de Waal, Frans B. M.; Brosnan, Sarah F.
2014-01-01
Humans will, at times, act against their own economic self-interest, for example, in gambling situations. To explore the evolutionary roots of this behavior, we modified a traditional human gambling task, the Iowa Gambling Task (IGT), for use with chimpanzees, capuchin monkeys and humans. We expanded the traditional task to include two additional payoff structures to fully elucidate the ways in which these primate species respond to differing reward distributions versus overall quantities of rewards, a component often missing in the existing literature. We found that while all three species respond as typical humans do in the standard IGT payoff structure, species and individual differences emerge in our new payoff structures. Specifically, when variance avoidance and reward maximization conflicted, roughly equivalent numbers of apes maximized their rewards and avoided variance, indicating that the traditional payoff structure of the IGT is insufficient to disentangle these competing strategies. Capuchin monkeys showed little consistency in their choices. To determine whether this was a true species difference or an effect of task presentation, we replicated the experiment but increased the intertrial interval. In this case, several capuchin monkeys followed a reward maximization strategy, while chimpanzees retained the same strategy they had used previously. This suggests that individual differences in strategies for interacting with variance and reward maximization are present in apes, but not in capuchin monkeys. The primate gambling task presented here is a useful methodology for disentangling strategies of variance avoidance and reward maximization. PMID:24504555
-
Role of delay-based reward in the spatial cooperation
NASA Astrophysics Data System (ADS)
Wang, Xu-Wen; Nie, Sen; Jiang, Luo-Luo; Wang, Bing-Hong; Chen, Shi-Ming
2017-01-01
Strategy selection in games, a typical decision making, usually brings noticeable reward for players which have discounted value if the delay appears. The discounted value is measure: earning sooner with a small reward or later with a delayed larger reward. Here, we investigate effects of delayed rewards on the cooperation in structured population. It is found that delayed reward supports the spreading of cooperation in square lattice, small-world and random networks. In particular, intermediate reward differences between delays impel the highest cooperation level. Interestingly, cooperative individuals with the same delay time steps form clusters to resist the invasion of defects, and cooperative individuals with lowest delay reward survive because they form the largest clusters in the lattice.
-
Pinto, Anthony; Steinglass, Joanna E.; Greene, Ashley L.; Weber, Elke U.; Simpson, H. Blair
2013-01-01
Background Although the relationship between obsessive compulsive disorder (OCD) and obsessive compulsive personality disorder (OCPD) has long been debated, clinical samples of OCD (without OCPD) and OCPD (without OCD) have never been systematically compared. We studied whether individuals with OCD, OCPD, or both conditions differ on symptomatology, functioning, and a measure of self-control: the capacity to delay reward. Methods 25 OCD, 25 OCPD, 25 comorbid OCD+OCPD, and 25 healthy controls (HC) completed clinical assessments and a validated intertemporal choice task that measures capacity to forego small immediate rewards for larger delayed rewards. Results OCD and OCPD subjects both showed impairment in psychosocial functioning and quality of life, as well as compulsive behavior, but only subjects with OCD reported obsessions. Individuals with OCPD, with or without comorbid OCD, discounted the value of delayed monetary rewards significantly less than OCD and HC. This excessive capacity to delay reward discriminates OCPD from OCD, and is associated with perfectionism and rigidity. Conclusions OCD and OCPD are both impairing disorders marked by compulsive behaviors, but they can be differentiated by the presence of obsessions in OCD and by excessive capacity to delay reward in OCPD. That individuals with OCPD show less temporal discounting (suggestive of excessive self-control) whereas prior studies have shown that individuals with substance use disorders show greater discounting (suggestive of impulsivity) supports the premise that this component of self-control lies on a continuum in which both extremes (impulsivity and overcontrol) contribute to psychopathology. PMID:24199665
-
Wickham, Robert J; Park, Jinwoo; Nunes, Eric J; Addy, Nii A
2015-08-12
Rapid, phasic dopamine (DA) release in the mammalian brain plays a critical role in reward processing, reinforcement learning, and motivational control. Fast scan cyclic voltammetry (FSCV) is an electrochemical technique with high spatial and temporal (sub-second) resolution that has been utilized to examine phasic DA release in several types of preparations. In vitro experiments in single-cells and brain slices and in vivo experiments in anesthetized rodents have been used to identify mechanisms that mediate dopamine release and uptake under normal conditions and in disease models. Over the last 20 years, in vivo FSCV experiments in awake, freely moving rodents have also provided insight of dopaminergic mechanisms in reward processing and reward learning. One major advantage of the awake, freely moving preparation is the ability to examine rapid DA fluctuations that are time-locked to specific behavioral events or to reward or cue presentation. However, one limitation of combined behavior and voltammetry experiments is the difficulty of dissociating DA effects that are specific to primary rewarding or aversive stimuli from co-occurring DA fluctuations that mediate reward-directed or other motor behaviors. Here, we describe a combined method using in vivo FSCV and intra-oral infusion in an awake rat to directly investigate DA responses to oral tastants. In these experiments, oral tastants are infused directly to the palate of the rat--bypassing reward-directed behavior and voluntary drinking behavior--allowing for direct examination of DA responses to tastant stimuli.
-
Williams, Leanne M; Gatt, Justine M; Hatch, Ainslie; Palmer, Donna M; Nagy, Marie; Rennie, Christopher; Cooper, Nicholas J; Morris, Charlotte; Grieve, Stuart; Dobson-Stone, Carol; Schofield, Peter; Clark, C Richard; Gordon, Evian; Arns, Martijn; Paul, Robert H
2008-09-01
This study was undertaken using the INTEGRATE Model of brain organization, which is based on a temporal continuum of emotion, thinking and self regulation. In this model, the key organizing principle of self adaption is the motivation to minimize danger and maximize reward. This principle drives brain organization across a temporal continuum spanning milliseconds to seconds, minutes and hours. The INTEGRATE Model comprises three distinct processes across this continuum. Emotion is defined by automatic action tendencies triggered by signals that are significant due to their relevance to minimizing danger-maximizing reward (such as abrupt, high contrast stimuli). Thinking represents cognitive functions and feelings that rely on brain and body feedback emerging from around 200 ms post-stimulus onwards. Self regulation is the modulation of emotion, thinking and feeling over time, according to more abstract adaptions to minimize danger-maximize reward. Here, we examined the impact of dispositional factors, age and genetic variation, on this temporal continuum. Brain Resource methodology provided a standardized platform for acquiring genetic, brain and behavioral data in the same 1000 healthy subjects. Results showed a "paradox" of declining function in the "thinking" time scale over the lifespan (6 to 80+ years), but a corresponding preservation or even increase in automatic functions of "emotion" and "self regulation". This paradox was paralleled by a greater loss of grey matter in cortical association areas (assessed using MRI) over age, but a relative preservation of subcortical grey matter. Genetic polymorphisms associated with both healthy function and susceptibility to disorder (including the BDNFVal(66)Met, COMTVal(158/108)Met, MAOA and DRD4 tandem repeat and 5HTT-LPR polymorphisms) made specific contributions to emotion, thinking and self regulatory functions, which also varied according to age.
-
Robust Encoding of Spatial Information in Orbitofrontal Cortex and Striatum.
Yoo, Seng Bum Michael; Sleezer, Brianna J; Hayden, Benjamin Y
2018-06-01
Knowing whether core reward regions carry information about the positions of relevant objects is crucial for adjudicating between choice models. One limitation of previous studies, including our own, is that spatial positions can be consistently differentially associated with rewards, and thus position can be confounded with attention, motor plans, or target identity. We circumvented these problems by using a task in which value-and thus choices-was determined solely by a frequently changing rule, which was randomized relative to spatial position on each trial. We presented offers asynchronously, which allowed us to control for reward expectation, spatial attention, and motor plans in our analyses. We find robust encoding of the spatial position of both offers and choices in two core reward regions, orbitofrontal Area 13 and ventral striatum, as well as in dorsal striatum of macaques. The trial-by-trial correlation in noise in encoding of position was associated with variation in choice, an effect known as choice probability correlation, suggesting that the spatial encoding is associated with choice and is not incidental to it. Spatial information and reward information are not carried by separate sets of neurons, although the two forms of information are temporally dissociable. These results highlight the ubiquity of multiplexed information in association cortex and argue against the idea that these ostensible reward regions serve as part of a pure value domain.
-
Scalzo, Franco; O’Connor, David A.; Orr, Catherine; Murphy, Kevin; Hester, Robert
2016-01-01
Deficits of self-control are associated with a number of mental state disorders. The ability to direct attention away from an alluring stimulus appears to aid inhibition of an impulsive response. However, further functional imaging research is required to assess the impact of shifts in attention on self-regulating processes. We varied the level of attentional disengagement in an functional magnetic resonance imaging (fMRI)-based Go/No-go task to probe whether diversion of attention away from alluring stimuli facilitates response inhibition. We used the attention-grabbing characteristic of faces to exogenously direct attention away from stimuli and investigated the relative importance of attention and response inhibition mechanisms under different delayed reward scenarios [i.e., where forgoing an immediate reward ($1) led to a higher ($10) or no payoff in the future]. We found that diverting attention improved response inhibition performance, but only when resistance to an alluring stimulus led to delayed reward. Region of interest analyses indicated significant increased activity in posterior right inferior frontal gyrus during successful No-go trials for delayed reward trials compared to no delayed reward trials, and significant reduction in activity in the superior temporal gyri and left caudate in contexts of high attentional diversion. Our findings imply that strategies that increase the perceived benefits of response inhibition might assist individuals in abstaining from problematic impulsive behaviors. PMID:27616988
-
Reward-dependent learning in neuronal networks for planning and decision making.
Dehaene, S; Changeux, J P
2000-01-01
Neuronal network models have been proposed for the organization of evaluation and decision processes in prefrontal circuitry and their putative neuronal and molecular bases. The models all include an implementation and simulation of an elementary reward mechanism. Their central hypothesis is that tentative rules of behavior, which are coded by clusters of active neurons in prefrontal cortex, are selected or rejected based on an evaluation by this reward signal, which may be conveyed, for instance, by the mesencephalic dopaminergic neurons with which the prefrontal cortex is densely interconnected. At the molecular level, the reward signal is postulated to be a neurotransmitter such as dopamine, which exerts a global modulatory action on prefrontal synaptic efficacies, either via volume transmission or via targeted synaptic triads. Negative reinforcement has the effect of destabilizing the currently active rule-coding clusters; subsequently, spontaneous activity varies again from one cluster to another, giving the organism the chance to discover and learn a new rule. Thus, reward signals function as effective selection signals that either maintain or suppress currently active prefrontal representations as a function of their current adequacy. Simulations of this variation-selection have successfully accounted for the main features of several major tasks that depend on prefrontal cortex integrity, such as the delayed-response test, the Wisconsin card sorting test, the Tower of London test and the Stroop test. For the more complex tasks, we have found it necessary to supplement the external reward input with a second mechanism that supplies an internal reward; it consists of an auto-evaluation loop which short-circuits the reward input from the exterior. This allows for an internal evaluation of covert motor intentions without actualizing them as behaviors, by simply testing them covertly by comparison with memorized former experiences. This element of architecture gives access to enhanced rates of learning via an elementary process of internal or covert mental simulation. We have recently applied these ideas to a new model, developed with M. Kerszberg, which hypothesizes that prefrontal cortex and its reward-related connections contribute crucially to conscious effortful tasks. This model distinguishes two main computational spaces within the human brain: a unique global workspace composed of distributed and heavily interconnected neurons with long-range axons, and a set of specialized and modular perceptual, motor, memory, evaluative and attentional processors. We postulate that workspace neurons are mobilized in effortful tasks for which the specialized processors do not suffice; they selectively mobilize or suppress, through descending connections, the contribution of specific processor neurons. In the course of task performance, workspace neurons become spontaneously co-activated, forming discrete though variable spatio-temporal patterns subject to modulation by vigilance signals and to selection by reward signals. A computer simulation of the Stroop task shows workspace activation to increase during acquisition of a novel task, effortful execution, and after errors. This model makes predictions concerning the spatio-temporal activation patterns during brain imaging of cognitive tasks, particularly concerning the conditions of activation of dorsolateral prefrontal cortex and anterior cingulate, their relation to reward mechanisms, and their specific reaction during error processing.
-
Context matters: Anterior and posterior cortical midline responses to sad movie scenes.
Schlochtermeier, L H; Pehrs, C; Bakels, J-H; Jacobs, A M; Kappelhoff, H; Kuchinke, L
2017-04-15
Narrative movies can create powerful emotional responses. While recent research has advanced the understanding of neural networks involved in immersive movie viewing, their modulation within a movie's dynamic context remains inconclusive. In this study, 24 healthy participants passively watched sad scene climaxes taken from 24 romantic comedies, while brain activity was measured using functional magnetic resonance (fMRI). To study effects of context, the sad scene climaxes were presented with either coherent scene context, replaced non-coherent context or without context. In a second viewing, the same clips were rated continuously for sadness. The ratings varied over time with peaks of experienced sadness within the assumed climax intervals. Activations in anterior and posterior cortical midline regions increased if presented with both coherent and replaced context, while activation in the temporal gyri decreased. This difference was more pronounced for the coherent context condition. Psycho-Physiological interactions (PPI) analyses showed a context-dependent coupling of midline regions with occipital visual and sub-cortical reward regions. Our results demonstrate the pivotal role of midline structures and their interaction with perceptual and reward areas in processing contextually embedded socio-emotional information in movies. Copyright © 2017 Elsevier B.V. All rights reserved.
-
Maass, Wolfgang
2008-01-01
Reward-modulated spike-timing-dependent plasticity (STDP) has recently emerged as a candidate for a learning rule that could explain how behaviorally relevant adaptive changes in complex networks of spiking neurons could be achieved in a self-organizing manner through local synaptic plasticity. However, the capabilities and limitations of this learning rule could so far only be tested through computer simulations. This article provides tools for an analytic treatment of reward-modulated STDP, which allows us to predict under which conditions reward-modulated STDP will achieve a desired learning effect. These analytical results imply that neurons can learn through reward-modulated STDP to classify not only spatial but also temporal firing patterns of presynaptic neurons. They also can learn to respond to specific presynaptic firing patterns with particular spike patterns. Finally, the resulting learning theory predicts that even difficult credit-assignment problems, where it is very hard to tell which synaptic weights should be modified in order to increase the global reward for the system, can be solved in a self-organizing manner through reward-modulated STDP. This yields an explanation for a fundamental experimental result on biofeedback in monkeys by Fetz and Baker. In this experiment monkeys were rewarded for increasing the firing rate of a particular neuron in the cortex and were able to solve this extremely difficult credit assignment problem. Our model for this experiment relies on a combination of reward-modulated STDP with variable spontaneous firing activity. Hence it also provides a possible functional explanation for trial-to-trial variability, which is characteristic for cortical networks of neurons but has no analogue in currently existing artificial computing systems. In addition our model demonstrates that reward-modulated STDP can be applied to all synapses in a large recurrent neural network without endangering the stability of the network dynamics. PMID:18846203
-
Incidental rewarding cues influence economic decisions in people with obesity.
Simmank, Jakob; Murawski, Carsten; Bode, Stefan; Horstmann, Annette
2015-01-01
Recent research suggests that obesity is linked to prominent alterations in learning and decision-making. This general difference may also underlie the preference for immediately consumable, highly palatable but unhealthy and high-calorie foods. Such poor food-related inter-temporal decision-making can explain weight gain; however, it is not yet clear whether this deficit can be generalized to other domains of inter-temporal decision-making, for example financial decisions. Further, little is known about the stability of decision-making behavior in obesity, especially in the presence of rewarding cues. To answer these questions, obese and lean participants (n = 52) completed two sessions of a novel priming paradigm including a computerized monetary delay discounting task. In the first session, general differences between groups in financial delay discounting were measured. In the second session, we tested the general stability of discount rates. Additionally, participants were primed by affective visual cues of different contextual categories before making financial decisions. We found that the obese group showed stronger discounting of future monetary rewards than the lean group, but groups did not differ in their general stability between sessions nor in their sensitivity toward changes in reward magnitude. In the obese group, a fast decrease of subjective value over time was directly related to a higher tendency for opportunistic eating. Obese in contrast to lean people were primed by the affective cues, showing a sex-specific pattern of priming direction. Our findings demonstrate that environments rich of cues, aiming at inducing unhealthy consumer decisions, can be highly detrimental for obese people. It also underscores that obesity is not merely a medical condition but has a strong cognitive component, meaning that current dietary and medical treatment strategies may fall too short.
-
Incidental rewarding cues influence economic decisions in people with obesity
Simmank, Jakob; Murawski, Carsten; Bode, Stefan; Horstmann, Annette
2015-01-01
Recent research suggests that obesity is linked to prominent alterations in learning and decision-making. This general difference may also underlie the preference for immediately consumable, highly palatable but unhealthy and high-calorie foods. Such poor food-related inter-temporal decision-making can explain weight gain; however, it is not yet clear whether this deficit can be generalized to other domains of inter-temporal decision-making, for example financial decisions. Further, little is known about the stability of decision-making behavior in obesity, especially in the presence of rewarding cues. To answer these questions, obese and lean participants (n = 52) completed two sessions of a novel priming paradigm including a computerized monetary delay discounting task. In the first session, general differences between groups in financial delay discounting were measured. In the second session, we tested the general stability of discount rates. Additionally, participants were primed by affective visual cues of different contextual categories before making financial decisions. We found that the obese group showed stronger discounting of future monetary rewards than the lean group, but groups did not differ in their general stability between sessions nor in their sensitivity toward changes in reward magnitude. In the obese group, a fast decrease of subjective value over time was directly related to a higher tendency for opportunistic eating. Obese in contrast to lean people were primed by the affective cues, showing a sex-specific pattern of priming direction. Our findings demonstrate that environments rich of cues, aiming at inducing unhealthy consumer decisions, can be highly detrimental for obese people. It also underscores that obesity is not merely a medical condition but has a strong cognitive component, meaning that current dietary and medical treatment strategies may fall too short. PMID:26528158
-
A Developmental Perspective on Neuroeconomic Mechanisms of Contingency Management
Stanger, Catherine; Budney, Alan J.; Bickel, Warren K.
2012-01-01
This paper provides a developmental overview of relevant theory and research on delay discounting and neuroeconomics, and their implications for CM approaches to treatment. Recent advances in neuroscience, and in particular the neuroscience of decision making, have the potential to inform treatment development for adolescent substance use in general, and contingency management (CM) treatments in particular. CM utilizes abstinence-based reinforcement to enhance motivation to engage in treatment and engender abstinence. CM interventions may be informed by research on delay discounting, a type of decision making that reflects how individuals value immediate vs. delayed rewards. Delay discounting reliably distinguishes substance abusers from non abusers and is a significant predictor of individual differences in response to substance use treatments. Delay discounting is also of high potential importance in the development of substance use problems in adolescence. Discounting may also be important in predicting response to CM, as CM attempts to directly influence this decision making process, shifting the preference from the immediate rewards of use to delayed rewards for choosing not to use. Multiple neural processes underlie decision making, and those processes have implications for adolescent substance abuse. There are significant neurodevelopmental processes that differentiate adolescents from adults. These processes are implicated in delay discounting, suggesting that adolescence may reflect a period of plasticity in temporal decision making. Understanding the neural mechanisms of delay discounting has led to promising working memory interventions directly targeting the executive functions that underlie individual choices. These interventions may be particularly helpful in combination with CM interventions that offer immediate rewards for brief periods of abstinence, and may show particular benefit in adolescence due to the heightened neural plasticity of systems that underlie temporal discounting in adolescence. PMID:22663343
-
Effects of emotion on prospection during decision-making.
Worthy, Darrell A; Byrne, Kaileigh A; Fields, Sherecce
2014-01-01
In two experiments we examined the role of emotion, specifically worry, anxiety, and mood, on prospection during decision-making. Worry is a particularly relevant emotion to study in the context of prospection because high levels of worry may make individuals more aversive toward the uncertainty associated with the prospect of obtaining future improvements in rewards or states. Thus, high levels of worry might lead to reduced prospection during decision-making and enhance preference for immediate over delayed rewards. In Experiment 1 participants performed a two-choice dynamic decision-making task where they were required to choose between one option (the decreasing option) which provided larger immediate rewards but declines in future states, and another option (the increasing option) which provided smaller immediate rewards but improvements in future states, making it the optimal choice. High levels of worry were associated with poorer performance in the task. Additionally, fits of a sophisticated reinforcement-learning model that incorporated both reward-based and state-based information suggested that individuals reporting high levels of worry gave greater weight to the immediate rewards they would receive on each trial than to the degree to which each action would lead to improvements in their future state. In Experiment 2 we found that high levels of worry were associated with greater delay discounting using a standard delay discounting task. Combined, the results suggest that high levels of worry are associated with reduced prospection during decision-making. We attribute these results to high worriers' aversion toward the greater uncertainty associated with attempting to improve future rewards than to maximize immediate reward. These results have implications for researchers interested in the effects of emotion on cognition, and suggest that emotion strongly affects the focus on temporal outcomes during decision-making.
-
Learning-dependent plasticity in human auditory cortex during appetitive operant conditioning.
Puschmann, Sebastian; Brechmann, André; Thiel, Christiane M
2013-11-01
Animal experiments provide evidence that learning to associate an auditory stimulus with a reward causes representational changes in auditory cortex. However, most studies did not investigate the temporal formation of learning-dependent plasticity during the task but rather compared auditory cortex receptive fields before and after conditioning. We here present a functional magnetic resonance imaging study on learning-related plasticity in the human auditory cortex during operant appetitive conditioning. Participants had to learn to associate a specific category of frequency-modulated tones with a reward. Only participants who learned this association developed learning-dependent plasticity in left auditory cortex over the course of the experiment. No differential responses to reward predicting and nonreward predicting tones were found in auditory cortex in nonlearners. In addition, learners showed similar learning-induced differential responses to reward-predicting and nonreward-predicting tones in the ventral tegmental area and the nucleus accumbens, two core regions of the dopaminergic neurotransmitter system. This may indicate a dopaminergic influence on the formation of learning-dependent plasticity in auditory cortex, as it has been suggested by previous animal studies. Copyright © 2012 Wiley Periodicals, Inc.
-
Anticipatory Reward Processing in Addicted Populations: A Focus on the Monetary Incentive Delay Task
Balodis, Iris M.; Potenza, Marc N.
2014-01-01
Advances in brain imaging techniques have allowed neurobiological research to temporally analyze signals coding for the anticipation of rewards. In addicted populations, both hypo- and hyper-responsiveness of brain regions (e.g., ventral striatum) implicated in drug effects and reward system processing have been reported during anticipation of generalized reward. Here, we discuss the current state of knowledge of reward processing in addictive disorders from a widely used and validated task: the Monetary Incentive Delay Task (MIDT). The current paper constrains review to those studies applying the MIDT in addicted and at-risk adult populations, with a focus on anticipatory processing and striatal regions activated during task performance, as well as the relationship of these regions with individual difference (e.g., impulsivity) and treatment outcome variables. We further review drug influences in challenge studies as a means to examine acute influences on reward processing in abstinent, recreationally using and addicted populations. Here, we discuss that generalized reward processing in addicted and at-risk populations is often characterized by divergent anticipatory signaling in the ventral striatum. Although methodological/task variations may underlie some discrepant findings, anticipatory signaling in the ventral striatum may also be influenced by smoking status, drug metabolites and treatment status in addicted populations. Divergent results across abstinent, recreationally using and addicted populations demonstrate complexities in interpreting findings. Future studies will benefit from focusing on characterizing how impulsivity and other addiction-related features relate to anticipatory striatal signaling over time. Additionally, identifying how anticipatory signals recover/adjust following protracted abstinence will be important in understanding recovery processes. PMID:25481621
-
Neural correlates of reward-based spatial learning in persons with cocaine dependence.
Tau, Gregory Z; Marsh, Rachel; Wang, Zhishun; Torres-Sanchez, Tania; Graniello, Barbara; Hao, Xuejun; Xu, Dongrong; Packard, Mark G; Duan, Yunsuo; Kangarlu, Alayar; Martinez, Diana; Peterson, Bradley S
2014-02-01
Dysfunctional learning systems are thought to be central to the pathogenesis of and impair recovery from addictions. The functioning of the brain circuits for episodic memory or learning that support goal-directed behavior has not been studied previously in persons with cocaine dependence (CD). Thirteen abstinent CD and 13 healthy participants underwent MRI scanning while performing a task that requires the use of spatial cues to navigate a virtual-reality environment and find monetary rewards, allowing the functional assessment of the brain systems for spatial learning, a form of episodic memory. Whereas both groups performed similarly on the reward-based spatial learning task, we identified disturbances in brain regions involved in learning and reward in CD participants. In particular, CD was associated with impaired functioning of medial temporal lobe (MTL), a brain region that is crucial for spatial learning (and episodic memory) with concomitant recruitment of striatum (which normally participates in stimulus-response, or habit, learning), and prefrontal cortex. CD was also associated with enhanced sensitivity of the ventral striatum to unexpected rewards but not to expected rewards earned during spatial learning. We provide evidence that spatial learning in CD is characterized by disturbances in functioning of an MTL-based system for episodic memory and a striatum-based system for stimulus-response learning and reward. We have found additional abnormalities in distributed cortical regions. Consistent with findings from animal studies, we provide the first evidence in humans describing the disruptive effects of cocaine on the coordinated functioning of multiple neural systems for learning and memory.
-
Pinto, Anthony; Steinglass, Joanna E; Greene, Ashley L; Weber, Elke U; Simpson, H Blair
2014-04-15
Although the relationship between obsessive-compulsive disorder (OCD) and obsessive-compulsive personality disorder (OCPD) has long been debated, clinical samples of OCD (without OCPD) and OCPD (without OCD) have never been systematically compared. We studied whether individuals with OCD, OCPD, or both conditions differ on symptomatology, functioning, and a measure of self-control: the capacity to delay reward. Twenty-five OCD, 25 OCPD, 25 comorbid OCD + OCPD, and 25 healthy control subjects completed clinical assessments and a validated intertemporal choice task that measures capacity to forego small immediate rewards for larger delayed rewards. OCD and OCPD subjects both showed impairment in psychosocial functioning and quality of life, as well as compulsive behavior, but only subjects with OCD reported obsessions. Individuals with OCPD, with or without comorbid OCD, discounted the value of delayed monetary rewards significantly less than OCD and healthy control subjects. This excessive capacity to delay reward discriminates OCPD from OCD and is associated with perfectionism and rigidity. OCD and OCPD are both impairing disorders marked by compulsive behaviors, but they can be differentiated by the presence of obsessions in OCD and by excessive capacity to delay reward in OCPD. That individuals with OCPD show less temporal discounting (suggestive of excessive self-control), whereas prior studies have shown that individuals with substance use disorders show greater discounting (suggestive of impulsivity), supports the premise that this component of self-control lies on a continuum in which both extremes (impulsivity and overcontrol) contribute to psychopathology. © 2013 Society of Biological Psychiatry Published by Society of Biological Psychiatry All rights reserved.
-
Release from proactive interference in rat spatial working memory.
Roberts, William A; MacDonald, Hayden; Brown, Lyn; Macpherson, Krista
2017-09-01
A three-phase procedure was used to produce proactive interference (PI) in one trial on an eight-arm radial maze. Rats were forced to enter four arms for reward on an initial interference phase, to then enter the four remaining arms on a target phase, and to then choose among all eight arms on a retention test, with only the arms not visited in the target phase containing reward. Control trials involved only the target phase and the retention test. Lower accuracy was found on PI trials than on control trials, but performance on PI trials significantly exceeded chance, showing some retention of target memories. Changes in temporal and reward variables between the interference, target, and retention test phases showed release from PI, but changes in context and pattern of arm entry did not. It is suggested that the release from PI paradigm can be used to understand spatial memory encoding in rats and other species.
-
Morey, Candice Coker; Cowan, Nelson; Morey, Richard D; Rouder, Jeffery N
2011-02-01
Prominent roles for general attention resources are posited in many models of working memory, but the manner in which these can be allocated differs between models or is not sufficiently specified. We varied the payoffs for correct responses in two temporally-overlapping recognition tasks, a visual array comparison task and a tone sequence comparison task. In the critical conditions, an increase in reward for one task corresponded to a decrease in reward for the concurrent task, but memory load remained constant. Our results show patterns of interference consistent with a trade-off between the tasks, suggesting that a shared resource can be flexibly divided, rather than only fully allotted to either of the tasks. Our findings support a role for a domain-general resource in models of working memory, and furthermore suggest that this resource is flexibly divisible.
-
Cross-talk between the epigenome and neural circuits in drug addiction.
Mews, Philipp; Calipari, Erin S
2017-01-01
Drug addiction is a behavioral disorder characterized by dysregulated learning about drugs and associated cues that result in compulsive drug seeking and relapse. Learning about drug rewards and predictive cues is a complex process controlled by a computational network of neural connections interacting with transcriptional and molecular mechanisms within each cell to precisely guide behavior. The interplay between rapid, temporally specific neuronal activation, and longer-term changes in transcription is of critical importance in the expression of appropriate, or in the case of drug addiction, inappropriate behaviors. Thus, these factors and their interactions must be considered together, especially in the context of treatment. Understanding the complex interplay between epigenetic gene regulation and circuit connectivity will allow us to formulate novel therapies to normalize maladaptive reward behaviors, with a goal of modulating addictive behaviors, while leaving natural reward-associated behavior unaffected. © 2017 Elsevier B.V. All rights reserved.
-
The impact of cognitive load on reward evaluation.
Krigolson, Olave E; Hassall, Cameron D; Satel, Jason; Klein, Raymond M
2015-11-19
The neural systems that afford our ability to evaluate rewards and punishments are impacted by a variety of external factors. Here, we demonstrate that increased cognitive load reduces the functional efficacy of a reward processing system within the human medial-frontal cortex. In our paradigm, two groups of participants used performance feedback to estimate the exact duration of one second while electroencephalographic (EEG) data was recorded. Prior to performing the time estimation task, both groups were instructed to keep their eyes still and avoid blinking in line with well established EEG protocol. However, during performance of the time-estimation task, one of the two groups was provided with trial-to-trial-feedback about their performance on the time-estimation task and their eye movements to induce a higher level of cognitive load relative to participants in the other group who were solely provided with feedback about the accuracy of their temporal estimates. In line with previous work, we found that the higher level of cognitive load reduced the amplitude of the feedback-related negativity, a component of the human event-related brain potential associated with reward evaluation within the medial-frontal cortex. Importantly, our results provide further support that increased cognitive load reduces the functional efficacy of a neural system associated with reward processing. Copyright © 2015 Elsevier B.V. All rights reserved.
-
A neuronal reward inequity signal in primate striatum
van Coeverden, Charlotte R.; Schultz, Wolfram
2015-01-01
Primates are social animals, and their survival depends on social interactions with others. Especially important for social interactions and welfare is the observation of rewards obtained by other individuals and the comparison with own reward. The fundamental social decision variable for the comparison process is reward inequity, defined by an asymmetric reward distribution among individuals. An important brain structure for coding reward inequity may be the striatum, a component of the basal ganglia involved in goal-directed behavior. Two rhesus monkeys were seated opposite each other and contacted a touch-sensitive table placed between them to obtain specific magnitudes of reward that were equally or unequally distributed among them. Response times in one of the animals demonstrated differential behavioral sensitivity to reward inequity. A group of neurons in the striatum showed distinct signals reflecting disadvantageous and advantageous reward inequity. These neuronal signals occurred irrespective of, or in conjunction with, own reward coding. These data demonstrate that striatal neurons of macaque monkeys sense the differences between other's and own reward. The neuronal activities are likely to contribute crucial reward information to neuronal mechanisms involved in social interactions. PMID:26378202
-
Sample path analysis of contribution and reward in cooperative groups.
Toyoizumi, Hiroshi
2009-02-07
Explaining cooperative behavior is one of the major challenges in both biology and human society. The individual reward in cooperative group depends on how we share the rewards in the group. Thus, the group size dynamics in a cooperative group and reward-allocation rule seem essential to evaluate the emergence of cooperative groups. We apply a sample path-based analysis called an extension of Little's formula to general cooperative group. We show that the expected reward is insensitive to the specific reward-allocation rule and probabilistic structure of group dynamics, and the simple productivity condition guarantees the expected reward to be larger than the average contribution. As an example, we take social queues to see the insensitivity result in detail.
-
Towards a general theory of neural computation based on prediction by single neurons.
Fiorillo, Christopher D
2008-10-01
Although there has been tremendous progress in understanding the mechanics of the nervous system, there has not been a general theory of its computational function. Here I present a theory that relates the established biophysical properties of single generic neurons to principles of Bayesian probability theory, reinforcement learning and efficient coding. I suggest that this theory addresses the general computational problem facing the nervous system. Each neuron is proposed to mirror the function of the whole system in learning to predict aspects of the world related to future reward. According to the model, a typical neuron receives current information about the state of the world from a subset of its excitatory synaptic inputs, and prior information from its other inputs. Prior information would be contributed by synaptic inputs representing distinct regions of space, and by different types of non-synaptic, voltage-regulated channels representing distinct periods of the past. The neuron's membrane voltage is proposed to signal the difference between current and prior information ("prediction error" or "surprise"). A neuron would apply a Hebbian plasticity rule to select those excitatory inputs that are the most closely correlated with reward but are the least predictable, since unpredictable inputs provide the neuron with the most "new" information about future reward. To minimize the error in its predictions and to respond only when excitation is "new and surprising," the neuron selects amongst its prior information sources through an anti-Hebbian rule. The unique inputs of a mature neuron would therefore result from learning about spatial and temporal patterns in its local environment, and by extension, the external world. Thus the theory describes how the structure of the mature nervous system could reflect the structure of the external world, and how the complexity and intelligence of the system might develop from a population of undifferentiated neurons, each implementing similar learning algorithms.
-
High Reward Makes Items Easier to Remember, but Harder to Bind to a New Temporal Context
Madan, Christopher R.; Fujiwara, Esther; Gerson, Bridgette C.; Caplan, Jeremy B.
2012-01-01
Learning through reward is central to adaptive behavior. Indeed, items are remembered better if they are experienced while participants expect a reward, and people can deliberately prioritize memory for high- over low-valued items. Do memory advantages for high-valued items only emerge after deliberate prioritization in encoding? Or, do reward-based memory enhancements also apply to unrewarded memory tests and to implicit memory? First, we tested for a high-value memory advantage in unrewarded implicit- and explicit-tests (Experiment 1). Participants first learned high or low-reward values of 36 words, followed by unrewarded lexical decision and free-recall tests. High-value words were judged faster in lexical decision, and more often recalled in free recall. These two memory advantages for high-value words were negatively correlated suggesting at least two mechanisms by which reward value can influence later item-memorability. The ease with which the values were originally acquired explained the negative correlation: people who learned values earlier showed reward effects in implicit memory whereas people who learned values later showed reward effects in explicit memory. We then asked whether a high-value advantage would persist if trained items were linked to a new context (Experiments 2a and 2b). Following the same value training as in Experiment 1, participants learned lists composed of previously trained words mixed with new words, each followed by free recall. Thus, participants had to retrieve words only from the most recent list, irrespective of their values. High- and low-value words were recalled equally, but low-value words were recalled earlier than high-value words and high-value words were more often intruded (proactive interference). Thus, the high-value advantage holds for implicit- and explicit-memory, but comes with a side effect: High-value items are more difficult to relearn in a new context. Similar to emotional arousal, reward value can both enhance and impair memory. PMID:22969711
-
Ventral pallidum roles in reward and motivation.
Smith, Kyle S; Tindell, Amy J; Aldridge, J Wayne; Berridge, Kent C
2009-01-23
In recent years the ventral pallidum has become a focus of great research interest as a mechanism of reward and incentive motivation. As a major output for limbic signals, the ventral pallidum was once associated primarily with motor functions rather than regarded as a reward structure in its own right. However, ample evidence now suggests that ventral pallidum function is a major mechanism of reward in the brain. We review data indicating that (1) an intact ventral pallidum is necessary for normal reward and motivation, (2) stimulated activation of ventral pallidum is sufficient to cause reward and motivation enhancements, and (3) activation patterns in ventral pallidum neurons specifically encode reward and motivation signals via phasic bursts of excitation to incentive and hedonic stimuli. We conclude that the ventral pallidum may serve as an important 'limbic final common pathway' for mesocorticolimbic processing of many rewards.
-
A comparison of the electrocortical response to monetary and social reward
Distefano, Amanda; Jackson, Felicia; Levinson, Amanda R; Infantolino, Zachary P; Jarcho, Johanna M; Nelson, Brady D
2018-01-01
Abstract Affective science research on reward processing has primarily focused on monetary rewards. There has been a growing interest in evaluating the neural basis of social decision-making and reward processing. The present study employed a within-subject design and compared the reward positivity (RewP), an event-related potential component that is present following favorable feedback and absent or reduced following unfavorable feedback, during monetary and social reward tasks. Specifically, 114 participants (75 females) completed a monetary reward task and a novel social reward task that were matched on trial structure, timing, and feedback stimuli in a counterbalanced order. Results indicated that the monetary and social RewP were of similar magnitude, positively correlated and demonstrated comparable psychometric properties, including reliability and dependability. Across both the monetary and social tasks, women demonstrated a greater RewP compared with men. This study provides a novel methodological approach toward examining the electrocortical response to social reward that is comparable to monetary reward. PMID:29373743
-
Herrera, David; Treviño, Mario
2015-01-01
In two-alternative discrimination tasks, experimenters usually randomize the location of the rewarded stimulus so that systematic behavior with respect to irrelevant stimuli can only produce chance performance on the learning curves. One way to achieve this is to use random numbers derived from a discrete binomial distribution to create a 'full random training schedule' (FRS). When using FRS, however, sporadic but long laterally-biased training sequences occur by chance and such 'input biases' are thought to promote the generation of laterally-biased choices (i.e., 'output biases'). As an alternative, a 'Gellerman-like training schedule' (GLS) can be used. It removes most input biases by prohibiting the reward from appearing on the same location for more than three consecutive trials. The sequence of past rewards obtained from choosing a particular discriminative stimulus influences the probability of choosing that same stimulus on subsequent trials. Assuming that the long-term average ratio of choices matches the long-term average ratio of reinforcers, we hypothesized that a reduced amount of input biases in GLS compared to FRS should lead to a reduced production of output biases. We compared the choice patterns produced by a 'Rational Decision Maker' (RDM) in response to computer-generated FRS and GLS training sequences. To create a virtual RDM, we implemented an algorithm that generated choices based on past rewards. Our simulations revealed that, although the GLS presented fewer input biases than the FRS, the virtual RDM produced more output biases with GLS than with FRS under a variety of test conditions. Our results reveal that the statistical and temporal properties of training sequences interacted with the RDM to influence the production of output biases. Thus, discrete changes in the training paradigms did not translate linearly into modifications in the pattern of choices generated by a RDM. Virtual RDMs could be further employed to guide the selection of proper training schedules for perceptual decision-making studies.
-
Cheng, Wei; Rolls, Edmund T; Qiu, Jiang; Liu, Wei; Tang, Yanqing; Huang, Chu-Chung; Wang, XinFa; Zhang, Jie; Lin, Wei; Zheng, Lirong; Pu, JunCai; Tsai, Shih-Jen; Yang, Albert C; Lin, Ching-Po; Wang, Fei; Xie, Peng; Feng, Jianfeng
2016-12-01
The first brain-wide voxel-level resting state functional connectivity neuroimaging analysis of depression is reported, with 421 patients with major depressive disorder and 488 control subjects. Resting state functional connectivity between different voxels reflects correlations of activity between those voxels and is a fundamental tool in helping to understand the brain regions with altered connectivity and function in depression. One major circuit with altered functional connectivity involved the medial orbitofrontal cortex Brodmann area 13, which is implicated in reward, and which had reduced functional connectivity in depression with memory systems in the parahippocampal gyrus and medial temporal lobe, especially involving the perirhinal cortex Brodmann area 36 and entorhinal cortex Brodmann area 28. The Hamilton Depression Rating Scale scores were correlated with weakened functional connectivity of the medial orbitofrontal cortex Brodmann area 13. Thus in depression there is decreased reward-related and memory system functional connectivity, and this is related to the depressed symptoms. The lateral orbitofrontal cortex Brodmann area 47/12, involved in non-reward and punishing events, did not have this reduced functional connectivity with memory systems. Second, the lateral orbitofrontal cortex Brodmann area 47/12 had increased functional connectivity with the precuneus, the angular gyrus, and the temporal visual cortex Brodmann area 21. This enhanced functional connectivity of the non-reward/punishment system (Brodmann area 47/12) with the precuneus (involved in the sense of self and agency), and the angular gyrus (involved in language) is thus related to the explicit affectively negative sense of the self, and of self-esteem, in depression. A comparison of the functional connectivity in 185 depressed patients not receiving medication and 182 patients receiving medication showed that the functional connectivity of the lateral orbitofrontal cortex Brodmann area 47/12 with these three brain areas was lower in the medicated than the unmedicated patients. This is consistent with the hypothesis that the increased functional connectivity of the lateral orbitofrontal cortex Brodmann area 47/12 is related to depression. Relating the changes in cortical connectivity to our understanding of the functions of different parts of the orbitofrontal cortex in emotion helps to provide new insight into the brain changes related to depression. © The Author (2016). Published by Oxford University Press on behalf of the Guarantors of Brain. All rights reserved. For Permissions, please email: journals.permissions@oup.com.
-
Smith, Bruce W; Mitchell, Derek G V; Hardin, Michael G; Jazbec, Sandra; Fridberg, Daniel; Blair, R James R; Ernst, Monique
2009-01-15
Economic decision-making involves the weighting of magnitude and probability of potential gains/losses. While previous work has examined the neural systems involved in decision-making, there is a need to understand how the parameters associated with decision-making (e.g., magnitude of expected reward, probability of expected reward and risk) modulate activation within these neural systems. In the current fMRI study, we modified the monetary wheel of fortune (WOF) task [Ernst, M., Nelson, E.E., McClure, E.B., Monk, C.S., Munson, S., Eshel, N., et al. (2004). Choice selection and reward anticipation: an fMRI study. Neuropsychologia 42(12), 1585-1597.] to examine in 25 healthy young adults the neural responses to selections of different reward magnitudes, probabilities, or risks. Selection of high, relative to low, reward magnitude increased activity in insula, amygdala, middle and posterior cingulate cortex, and basal ganglia. Selection of low-probability, as opposed to high-probability reward, increased activity in anterior cingulate cortex, as did selection of risky, relative to safe reward. In summary, decision-making that did not involve conflict, as in the magnitude contrast, recruited structures known to support the coding of reward values, and those that integrate motivational and perceptual information for behavioral responses. In contrast, decision-making under conflict, as in the probability and risk contrasts, engaged the dorsal anterior cingulate cortex whose role in conflict monitoring is well established. However, decision-making under conflict failed to activate the structures that track reward values per se. Thus, the presence of conflict in decision-making seemed to significantly alter the pattern of neural responses to simple rewards. In addition, this paradigm further clarifies the functional specialization of the cingulate cortex in processes of decision-making.
-
Abnormal Temporal Difference Reward-Learning Signals in Major Depression
ERIC Educational Resources Information Center
Kumar, P.; Waiter, G.; Ahearn, T.; Milders, M.; Reid, I.; Steele, J. D.
2008-01-01
Anhedonia is a core symptom of major depressive disorder (MDD), long thought to be associated with reduced dopaminergic function. However, most antidepressants do not act directly on the dopamine system and all antidepressants have a delayed full therapeutic effect. Recently, it has been proposed that antidepressants fail to alter dopamine…
-
ERIC Educational Resources Information Center
Rassuli, Ali
2012-01-01
Extrinsic inducements to adjust students' learning motivations have evolved within 2 opposing paradigms. Cognitive evaluation theories claim that controlling factors embedded in extrinsic rewards dissipate intrinsic aspirations. Behavioral theorists contend that if engagement is voluntary, extrinsic reinforcements enhance learning without ill…
-
Decision processes in choice overload: a product of delay and probability discounting?
Kaplan, Brent A; Reed, Derek D
2013-07-01
Recent research in the behavioral decision making literature has demonstrated that humans hyperbolically discount the subjective value of options as the number of options increases (Reed et al., 2012). These findings provide a cognitive-behavioral synthesis of the "choice overload" phenomenon, also known as the "paradox of choice." Specifically, these findings suggest that temporal discounting may serve as the underlying process contributing to this effect. As an extension, this study examined the effects of reward magnitude sizes had on rates temporal and options discounting. This manipulation was conducted to determine what role temporal discounting plays in discounting of options. The present results suggest that temporal discounting may not be the only process contributing to the choice overload effect. Copyright © 2013 Elsevier B.V. All rights reserved.
-
Do the Powerful Discount the Future Less? The Effects of Power on Temporal Discounting
Duan, Jinyun; Wu, Sherry J.; Sun, Luying
2017-01-01
Individuals have the tendency to discount rewards in the future, known as temporal discounting, and we find that sense of power (the felt capacity to influence the thinking and behavior of others) reduces such tendency. In Studies 1 and 2, we used both an experiment and a survey with organizational employees to demonstrate that power reduced temporal discounting. In Study 3, we replicated study 1 while exploring a unique cultural trait of Danbo, or indifference to fame and wealth, across two ethnic groups (Han and Tibetan groups) in China. While power reduces temporal discounting, the relationship between the two may be leveraged by individual differences of optimism, frustration, and Danbo. The results imply a more nuanced interpretation of how individual and situational factors can affect intertemporal choice. PMID:28680410
-
Do the Powerful Discount the Future Less? The Effects of Power on Temporal Discounting.
Duan, Jinyun; Wu, Sherry J; Sun, Luying
2017-01-01
Individuals have the tendency to discount rewards in the future, known as temporal discounting, and we find that sense of power (the felt capacity to influence the thinking and behavior of others) reduces such tendency. In Studies 1 and 2, we used both an experiment and a survey with organizational employees to demonstrate that power reduced temporal discounting. In Study 3, we replicated study 1 while exploring a unique cultural trait of Danbo , or indifference to fame and wealth, across two ethnic groups (Han and Tibetan groups) in China. While power reduces temporal discounting, the relationship between the two may be leveraged by individual differences of optimism, frustration, and Danbo . The results imply a more nuanced interpretation of how individual and situational factors can affect intertemporal choice.
-
Salama, Aallaa; Gründer, Gerhard; Spreckelmeyer, Katja N.
2014-01-01
Recent studies have reported inconsistent results regarding the loss of reward sensitivity in the aging brain. Although such an age effect might be due to a decline of physiological processes, it may also be a consequence of age-related changes in motivational preference for different rewards. Here, we examined whether the age effects on neural correlates of reward anticipation are modulated by the type of expected reward. Functional magnetic resonance images were acquired in 24 older (60–78 years) and 24 young participants (20–28 years) while they performed an incentive delay task offering monetary or social rewards. Anticipation of either reward type recruited brain structures associated with reward, including the nucleus accumbens (NAcc). Region of interest analysis revealed an interaction effect of reward type and age group in the right NAcc: enhanced activation to cues of social reward was detected in the older subsample while enhanced activation to cues of monetary reward was detected in the younger subsample. Our results suggest that neural sensitivity to reward-predicting cues does not generally decrease with age. Rather, neural responses in the NAcc appear to be modulated by the type of reward, presumably reflecting age-related changes in motivational value attributed to different types of reward. PMID:23547243
-
Neuropharmacological mechanisms of drug reward: beyond dopamine in the nucleus accumbens.
Bardo, M T
1998-01-01
Multiple lines of research have implicated the mesolimbic dopamine system in drug reward measured by either the drug self-administration or conditioned place preference paradigm. The present review summarizes recent work that examines the neuropharmacological mechanisms by which drugs impinge on this dopaminergic neural circuitry, as well as other systems that provide input and output circuits to the mesolimbic dopamine system. Studies examining the effect of selective agonist and antagonist drugs administered systemically have indicated that multiple neurotransmitters are involved, including dopamine, serotonin, acetylcholine, glutamate, GABA, and various peptides. Direct microinjection studies have also provided crucial evidence indicating that, in addition to the mesolimbic dopamine system, other structures play a role in drug reward, including the ventral pallidum, amygdala, hippocampus, hypothalamus, and pedunculopontine tegmental nucleus. GABAergic circuitry descending from the nucleus accumbens to the pedunculopontine tegmental nucleus via the ventral pallidum appears to be especially important in directing the behavioral sequelae associated with reward produced by various drugs of abuse. However, activation of the reward circuitry is achieved differently for various drugs of abuse. With amphetamine and cocaine, initiation of reward is controlled within the nucleus accumbens and prefrontal cortex, respectively. With opiates, initiation of reward involves the ventral tegmental area, nucleus accumbens, hippocampus, and hypothalamus. It is not clear presently if these multiple anatomical structures mediate opiate reward by converging on a single output system or multiple output systems.
-
Mechanisms of value-learning in the guidance of spatial attention.
Anderson, Brian A; Kim, Haena
2018-05-11
The role of associative reward learning in the guidance of feature-based attention is well established. The extent to which reward learning can modulate spatial attention has been much more controversial. At least one demonstration of a persistent spatial attention bias following space-based associative reward learning has been reported. At the same time, multiple other experiments have been published failing to demonstrate enduring attentional biases towards locations at which a target, if found, yields high reward. This is in spite of evidence that participants use reward structures to inform their decisions where to search, leading some to suggest that, unlike feature-based attention, spatial attention may be impervious to the influence of learning from reward structures. Here, we demonstrate a robust bias towards regions of a scene that participants were previously rewarded for selecting. This spatial bias relies on representations that are anchored to the configuration of objects within a scene. The observed bias appears to be driven specifically by reinforcement learning, and can be observed with equal strength following non-reward corrective feedback. The time course of the bias is consistent with a transient shift of attention, rather than a strategic search pattern, and is evident in eye movement patterns during free viewing. Taken together, our findings reconcile previously conflicting reports and offer an integrative account of how learning from feedback shapes the spatial attention system. Copyright © 2018 Elsevier B.V. All rights reserved.
-
Ventral Pallidum Roles in Reward and Motivation
Smith, Kyle S.; Tindell, Amy J.; Aldridge, J. Wayne; Berridge, Kent C.
2008-01-01
In recent years the ventral pallidum has become a focus of great research interest as a mechanism of reward and incentive motivation. As a major output for limbic signals, the ventral pallidum was once associated primarily with motor functions rather than regarded as a reward structure in its own right. However, ample evidence now suggests that ventral pallidum function is a major mechanism of reward in the brain. We review data indicating that 1) an intact ventral pallidum is necessary for normal reward and motivation, 2) stimulated activation of ventral pallidum is sufficient to cause reward and motivation enhancements, and 3) activation patterns in ventral pallidum neurons specifically encode reward and motivation signals via phasic bursts of excitation to incentive and hedonic stimuli. We conclude that the ventral pallidum may serve as an important ‘limbic final common pathway’ for mesocorticolimbic processing of many rewards. PMID:18955088
-
Diekhof, Esther K; Keil, Maria; Obst, Katrin U; Henseler, Ilona; Dechent, Peter; Falkai, Peter; Gruber, Oliver
2012-09-14
There is ample evidence of gender differences in neural processes and behavior. Differences in reward-related behaviors have been linked to either temporary or permanent organizational influences of gonadal hormones on the mesolimbic dopamine system and reward-related activation. Still, little is known about the association between biological gender and the neural underpinnings of the ability to resist reward-related impulses. Here we assessed with functional magnetic resonance imaging which neural processes enable men and women to successfully control their desire for immediate reward when this is required by a higher-order goal (i.e., during a 'desire-reason dilemma'; Diekhof and Gruber, 2010). Thirty-two participants (16 females) were closely matched for age, personality characteristics (e.g., novelty seeking) and behavioral performance in the 'desire-reason task'. On the neural level, men and women showed similarities in the general response of the nucleus accumbens and of the ventral tegmental area to predictors of immediate reward, but they differed in additional brain mechanisms that enabled self-controlled decisions against the preference for immediate reward. Firstly, men exhibited a stronger reduction of activation in the ventral pallidum, putamen, temporal pole and pregenual anterior cingulate cortex during the 'desire-reason dilemma'. Secondly, connectivity analyses revealed a significant change in the direction of the connectivity between anteroventral prefrontal cortex and nucleus accumbens during decisions counteracting the reward-related impulse when comparing men and women. Together, these findings support the view of a sexual dimorphism that manifested in the recruitment of gender-specific neural resources during the successful deployment of self-control. Copyright © 2012 Elsevier B.V. All rights reserved.
-
Balodis, Iris M; Potenza, Marc N
2015-03-01
Advances in brain imaging techniques have allowed neurobiological research to temporally analyze signals coding for the anticipation of reward. In addicted populations, both hyporesponsiveness and hyperresponsiveness of brain regions (e.g., ventral striatum) implicated in drug effects and reward system processing have been reported during anticipation of generalized reward. We discuss the current state of knowledge of reward processing in addictive disorders from a widely used and validated task: the monetary incentive delay task. Only studies applying the monetary incentive delay task in addicted and at-risk adult populations are reviewed, with a focus on anticipatory processing and striatal regions activated during task performance as well as the relationship of these regions with individual difference (e.g., impulsivity) and treatment outcome variables. We further review drug influences in challenge studies as a means to examine acute influences on reward processing in abstinent, recreationally using, and addicted populations. Generalized reward processing in addicted and at-risk populations is often characterized by divergent anticipatory signaling in the ventral striatum. Although methodologic and task variations may underlie some discrepant findings, anticipatory signaling in the ventral striatum may also be influenced by smoking status, drug metabolites, and treatment status in addicted populations. Divergent results across abstinent, recreationally using, and addicted populations demonstrate complexities in interpreting findings. Future studies would benefit from focusing on characterizing how impulsivity and other addiction-related features relate to anticipatory striatal signaling over time. Additionally, identifying how anticipatory signals recover or adjust after protracted abstinence will be important in understanding recovery processes. Copyright © 2015 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.
-
Stereotype Threat Reinterpreted as a Regulatory Mismatch
Grimm, Lisa R.; Markman, Arthur B.; Maddox, W. Todd; Baldwin, Grant C.
2008-01-01
Research documents performance decrements resulting from the activation of a negative task-relevant stereotype. We combine a number of strands of work to identify causes of stereotype threat in a way that allows us to reverse the effects and improve the performance of individuals with negative task-relevant stereotypes. We draw on prior work suggesting that negative stereotypes induce a prevention focus, and other research suggesting that people exhibit greater flexibility when their regulatory focus matches the reward structure of the task. This work suggests that stereotype threat effects emerge from a prevention focus combined with tasks that have an explicit or implicit gains reward structure. We find flexible performance can be induced in individuals who have a negative task-relevant stereotype by using a losses reward structure. We demonstrate the interaction of stereotypes and the reward structure of the task using chronic stereotypes and GRE math problems (Experiment 1), and primed stereotypes and a category learning task (Experiments 2a and 2b). We discuss implications of this research for other work on stereotype threat. PMID:19159133
-
Probability matching in perceptrons: Effects of conditional dependence and linear nonseparability.
Dawson, Michael R W; Gupta, Maya
2017-01-01
Probability matching occurs when the behavior of an agent matches the likelihood of occurrence of events in the agent's environment. For instance, when artificial neural networks match probability, the activity in their output unit equals the past probability of reward in the presence of a stimulus. Our previous research demonstrated that simple artificial neural networks (perceptrons, which consist of a set of input units directly connected to a single output unit) learn to match probability when presented different cues in isolation. The current paper extends this research by showing that perceptrons can match probabilities when presented simultaneous cues, with each cue signaling different reward likelihoods. In our first simulation, we presented up to four different cues simultaneously; the likelihood of reward signaled by the presence of one cue was independent of the likelihood of reward signaled by other cues. Perceptrons learned to match reward probabilities by treating each cue as an independent source of information about the likelihood of reward. In a second simulation, we violated the independence between cues by making some reward probabilities depend upon cue interactions. We did so by basing reward probabilities on a logical combination (AND or XOR) of two of the four possible cues. We also varied the size of the reward associated with the logical combination. We discovered that this latter manipulation was a much better predictor of perceptron performance than was the logical structure of the interaction between cues. This indicates that when perceptrons learn to match probabilities, they do so by assuming that each signal of a reward is independent of any other; the best predictor of perceptron performance is a quantitative measure of the independence of these input signals, and not the logical structure of the problem being learned.
-
Probability matching in perceptrons: Effects of conditional dependence and linear nonseparability
2017-01-01
Probability matching occurs when the behavior of an agent matches the likelihood of occurrence of events in the agent’s environment. For instance, when artificial neural networks match probability, the activity in their output unit equals the past probability of reward in the presence of a stimulus. Our previous research demonstrated that simple artificial neural networks (perceptrons, which consist of a set of input units directly connected to a single output unit) learn to match probability when presented different cues in isolation. The current paper extends this research by showing that perceptrons can match probabilities when presented simultaneous cues, with each cue signaling different reward likelihoods. In our first simulation, we presented up to four different cues simultaneously; the likelihood of reward signaled by the presence of one cue was independent of the likelihood of reward signaled by other cues. Perceptrons learned to match reward probabilities by treating each cue as an independent source of information about the likelihood of reward. In a second simulation, we violated the independence between cues by making some reward probabilities depend upon cue interactions. We did so by basing reward probabilities on a logical combination (AND or XOR) of two of the four possible cues. We also varied the size of the reward associated with the logical combination. We discovered that this latter manipulation was a much better predictor of perceptron performance than was the logical structure of the interaction between cues. This indicates that when perceptrons learn to match probabilities, they do so by assuming that each signal of a reward is independent of any other; the best predictor of perceptron performance is a quantitative measure of the independence of these input signals, and not the logical structure of the problem being learned. PMID:28212422
-
Namboodiri, Vijay Mohan K; Levy, Joshua M; Mihalas, Stefan; Sims, David W; Hussain Shuler, Marshall G
2016-08-02
Understanding the exploration patterns of foragers in the wild provides fundamental insight into animal behavior. Recent experimental evidence has demonstrated that path lengths (distances between consecutive turns) taken by foragers are well fitted by a power law distribution. Numerous theoretical contributions have posited that "Lévy random walks"-which can produce power law path length distributions-are optimal for memoryless agents searching a sparse reward landscape. It is unclear, however, whether such a strategy is efficient for cognitively complex agents, from wild animals to humans. Here, we developed a model to explain the emergence of apparent power law path length distributions in animals that can learn about their environments. In our model, the agent's goal during search is to build an internal model of the distribution of rewards in space that takes into account the cost of time to reach distant locations (i.e., temporally discounting rewards). For an agent with such a goal, we find that an optimal model of exploration in fact produces hyperbolic path lengths, which are well approximated by power laws. We then provide support for our model by showing that humans in a laboratory spatial exploration task search space systematically and modify their search patterns under a cost of time. In addition, we find that path length distributions in a large dataset obtained from free-ranging marine vertebrates are well described by our hyperbolic model. Thus, we provide a general theoretical framework for understanding spatial exploration patterns of cognitively complex foragers.
-
Layered reward signalling through octopamine and dopamine in Drosophila.
Burke, Christopher J; Huetteroth, Wolf; Owald, David; Perisse, Emmanuel; Krashes, Michael J; Das, Gaurav; Gohl, Daryl; Silies, Marion; Certel, Sarah; Waddell, Scott
2012-12-20
Dopamine is synonymous with reward and motivation in mammals. However, only recently has dopamine been linked to motivated behaviour and rewarding reinforcement in fruitflies. Instead, octopamine has historically been considered to be the signal for reward in insects. Here we show, using temporal control of neural function in Drosophila, that only short-term appetitive memory is reinforced by octopamine. Moreover, octopamine-dependent memory formation requires signalling through dopamine neurons. Part of the octopamine signal requires the α-adrenergic-like OAMB receptor in an identified subset of mushroom-body-targeted dopamine neurons. Octopamine triggers an increase in intracellular calcium in these dopamine neurons, and their direct activation can substitute for sugar to form appetitive memory, even in flies lacking octopamine. Analysis of the β-adrenergic-like OCTβ2R receptor reveals that octopamine-dependent reinforcement also requires an interaction with dopamine neurons that control appetitive motivation. These data indicate that sweet taste engages a distributed octopamine signal that reinforces memory through discrete subsets of mushroom-body-targeted dopamine neurons. In addition, they reconcile previous findings with octopamine and dopamine and suggest that reinforcement systems in flies are more similar to mammals than previously thought.
-
Reward uncertainty enhances incentive salience attribution as sign-tracking
Anselme, Patrick; Robinson, Mike J. F.; Berridge, Kent C.
2014-01-01
Conditioned stimuli (CSs) come to act as motivational magnets following repeated association with unconditioned stimuli (UCSs) such as sucrose rewards. By traditional views, the more reliably predictive a Pavlovian CS-UCS association, the more the CS becomes attractive. However, in some cases, less predictability might equal more motivation. Here we examined the effect of introducing uncertainty in CS-UCS association on CS strength as an attractive motivation magnet. In the present study, Experiment 1 assessed the effects of Pavlovian predictability versus uncertainty about reward probability and/or reward magnitude on the acquisition and expression of sign-tracking (ST) and goal-tracking (GT) responses in an autoshaping procedure. Results suggested that uncertainty produced strongest incentive salience expressed as sign-tracking. Experiment 2 examined whether a within-individual temporal shift from certainty to uncertainty conditions could produce a stronger CS motivational magnet when uncertainty began, and found that sign-tracking still increased after the shift. Overall, our results support earlier reports that ST responses become more pronounced in the presence of uncertainty regarding CS-UCS associations, especially when uncertainty combines both probability and magnitude. These results suggest that Pavlovian uncertainty, although diluting predictability, is still able to enhance the incentive motivational power of particular CSs. PMID:23078951
-
Reward magnitude tracking by neural populations in ventral striatum
Fiallos, Ana M.; Bricault, Sarah J.; Cai, Lili X.; Worku, Hermoon A.; Colonnese, Matthew T.; Westmeyer, Gil; Jasanoff, Alan
2017-01-01
Evaluation of the magnitudes of intrinsically rewarding stimuli is essential for assigning value and guiding behavior. By combining parametric manipulation of a primary reward, medial forebrain bundle (MFB) microstimulation, with functional magnetic imaging (fMRI) in rodents, we delineated a broad network of structures activated by behaviorally characterized levels of rewarding stimulation. Correlation of psychometric behavioral measurements with fMRI response magnitudes revealed regions whose activity corresponded closely to the subjective magnitude of rewards. The largest and most reliable focus of reward magnitude tracking was observed in the shell region of the nucleus accumbens (NAc). Although the nonlinear nature of neurovascular coupling complicates interpretation of fMRI findings in precise neurophysiological terms, reward magnitude tracking was not observed in vascular compartments and could not be explained by saturation of region-specific hemodynamic responses. In addition, local pharmacological inactivation of NAc changed the profile of animals’ responses to rewards of different magnitudes without altering mean reward response rates, further supporting a hypothesis that neural population activity in this region contributes to assessment of reward magnitudes. PMID:27789262
-
Dopaminergic modulation of the trade-off between probability and time in economic decision-making.
Arrondo, Gonzalo; Aznárez-Sanado, Maite; Fernández-Seara, Maria A; Goñi, Joaquín; Loayza, Francis R; Salamon-Klobut, Ewa; Heukamp, Franz H; Pastor, Maria A
2015-06-01
Studies on animals and humans have demonstrated the importance of dopamine in modulating decision-making processes. In this work, we have tested dopaminergic modulation of economic decision-making and its neural correlates by administering either placebo or metoclopramide, a dopamine D2-receptor antagonist, to healthy subjects, during a functional MRI study. The decision-making task combined probability and time delay with a fixed monetary reward. For individual behavioral characterization, we used the Probability Time Trade-off (PTT) economic model, which integrates the traditional trade-offs of reward magnitude-time and reward magnitude-probability into a single measurement, thereby quantifying the subjective value of a delayed and probabilistic outcome. A regression analysis between BOLD signal and the PTT model index permitted to identify the neural substrate encoding the subjective reward-value. Behaviorally, medication reduced the rate of temporal discounting over probability, reflected in medicated subjects being more prone to postpone the reward in order to increase the outcome probability. In addition, medicated subjects showed less activity during the task in the postcentral gyrus as well as frontomedian areas, whereas there were no differences in the ventromedial orbitofrontal cortex (VMOFC) between groups when coding the subjective value. The present study demonstrates by means of behavior and imaging that dopamine modulation alters the probability-time trade-off in human economic decision-making. Copyright © 2015 Elsevier B.V. and ECNP. All rights reserved.
-
The Intertemporal Stability of Teacher Effect Estimates. Working Paper 2008-22
ERIC Educational Resources Information Center
McCaffrey, Daniel F.; Sass, Tim R.; Lockwood, J.R.
2008-01-01
Recently, a number of school districts have begun using measures of teachers' contributions to student test scores or teacher "value added" to determine salaries and other monetary rewards. In this paper we investigate the precision of value-added measures by analyzing their inter-temporal stability. We find that these measures of…
-
Prefrontal Cortex and Impulsive Decision Making
Kim, Soyoun; Lee, Daeyeol
2010-01-01
Impulsivity refers to a set of heterogeneous behaviors that are tuned suboptimally along certain temporal dimensions. Impulsive inter-temporal choice refers to the tendency to forego a large but delayed reward and to seek an inferior but more immediate reward, whereas impulsive motor responses also result when the subjects fail to suppress inappropriate automatic behaviors. In addition, impulsive actions can be produced when too much emphasis is placed on speed rather than accuracy in a wide range of behaviors, including perceptual decision making. Despite this heterogeneous nature, the prefrontal cortex and its connected areas, such as the basal ganglia, play an important role in gating impulsive actions in a variety of behavioral tasks. Here, we describe key features of computations necessary for optimal decision making, and how their failures can lead to impulsive behaviors. We also review the recent findings from neuroimaging and single-neuron recording studies on the neural mechanisms related to impulsive behaviors. Converging approaches in economics, psychology, and neuroscience provide a unique vista for better understanding the nature of behavioral impairments associated with impulsivity. PMID:20728878
-
Scherbaum, Stefan; Dshemuchadse, Maja; Goschke, Thomas
2012-01-01
Temporal discounting denotes the fact that individuals prefer smaller rewards delivered sooner over larger rewards delivered later, often to a higher extent than suggested by normative economical theories. In this article, we identify three lines of research studying this phenomenon which aim (i) to describe temporal discounting mathematically, (ii) to explain observed choice behavior psychologically, and (iii) to predict the influence of specific factors on intertemporal decisions. We then opt for an approach integrating postulated mechanisms and empirical findings from these three lines of research. Our approach focuses on the dynamical properties of decision processes and is based on computational modeling. We present a dynamic connectionist model of intertemporal choice focusing on the role of self-control and time framing as two central factors determining choice behavior. Results of our simulations indicate that the two influences interact with each other, and we present experimental data supporting this prediction. We conclude that computational modeling of the decision process dynamics can advance the integration of different strands of research in intertemporal choice. PMID:23181048
-
Scherbaum, Stefan; Dshemuchadse, Maja; Goschke, Thomas
2012-01-01
Temporal discounting denotes the fact that individuals prefer smaller rewards delivered sooner over larger rewards delivered later, often to a higher extent than suggested by normative economical theories. In this article, we identify three lines of research studying this phenomenon which aim (i) to describe temporal discounting mathematically, (ii) to explain observed choice behavior psychologically, and (iii) to predict the influence of specific factors on intertemporal decisions. We then opt for an approach integrating postulated mechanisms and empirical findings from these three lines of research. Our approach focuses on the dynamical properties of decision processes and is based on computational modeling. We present a dynamic connectionist model of intertemporal choice focusing on the role of self-control and time framing as two central factors determining choice behavior. Results of our simulations indicate that the two influences interact with each other, and we present experimental data supporting this prediction. We conclude that computational modeling of the decision process dynamics can advance the integration of different strands of research in intertemporal choice.
-
Rademacher, Lena; Salama, Aallaa; Gründer, Gerhard; Spreckelmeyer, Katja N
2014-06-01
Recent studies have reported inconsistent results regarding the loss of reward sensitivity in the aging brain. Although such an age effect might be due to a decline of physiological processes, it may also be a consequence of age-related changes in motivational preference for different rewards. Here, we examined whether the age effects on neural correlates of reward anticipation are modulated by the type of expected reward. Functional magnetic resonance images were acquired in 24 older (60-78 years) and 24 young participants (20-28 years) while they performed an incentive delay task offering monetary or social rewards. Anticipation of either reward type recruited brain structures associated with reward, including the nucleus accumbens (NAcc). Region of interest analysis revealed an interaction effect of reward type and age group in the right NAcc: enhanced activation to cues of social reward was detected in the older subsample while enhanced activation to cues of monetary reward was detected in the younger subsample. Our results suggest that neural sensitivity to reward-predicting cues does not generally decrease with age. Rather, neural responses in the NAcc appear to be modulated by the type of reward, presumably reflecting age-related changes in motivational value attributed to different types of reward. © The Author (2013). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
Impaired Learning of Social Compared to Monetary Rewards in Autism
Lin, Alice; Rangel, Antonio; Adolphs, Ralph
2012-01-01
A leading hypothesis to explain the social dysfunction in people with autism spectrum disorders (ASD) is that they exhibit a deficit in reward processing and motivation specific to social stimuli. However, there have been few direct tests of this hypothesis to date. Here we used an instrumental reward learning task that contrasted learning with social rewards (pictures of positive and negative faces) against learning with monetary reward (winning and losing money). The two tasks were structurally identical except for the type of reward, permitting direct comparisons. We tested 10 high-functioning people with ASD (7M, 3F) and 10 healthy controls who were matched on gender, age, and education. We found no significant differences between the two groups in terms of overall ability behaviorally to discriminate positive from negative slot machines, reaction-times, and valence ratings, However, there was a specific impairment in the ASD group in learning to choose social rewards, compared to monetary rewards: they had a significantly lower cumulative number of choices of the most rewarding social slot machine, and had a significantly slower initial learning rate for the socially rewarding slot machine, compared to the controls. The findings show a deficit in reward learning in ASD that is greater for social rewards than for monetary rewards, and support the hypothesis of a disproportionate impairment in social reward processing in ASD. PMID:23060743
-
Gheza, Davide; Paul, Katharina; Pourtois, Gilles
2017-11-24
Evaluative feedback provided during performance monitoring (PM) elicits either a positive or negative deflection ~250-300ms after its onset in the event-related potential (ERP) depending on whether the outcome is reward-related or not, as well as expected or not. However, it remains currently unclear whether these two deflections reflect a unitary process, or rather dissociable effects arising from non-overlapping brain networks. To address this question, we recorded 64-channel EEG in healthy adult participants performing a standard gambling task where valence and expectancy were manipulated in a factorial design. We analyzed the feedback-locked ERP data using a conventional ERP analysis, as well as an advanced topographic ERP mapping analysis supplemented with distributed source localization. Results reveal two main topographies showing opposing valence effects, and being differently modulated by expectancy. The first one was short-lived and sensitive to no-reward irrespective of expectancy. Source-estimation associated with this topographic map comprised mainly regions of the dorsal anterior cingulate cortex. The second one was primarily driven by reward, had a prolonged time-course and was monotonically influenced by expectancy. Moreover, this reward-related topographical map was best accounted for by intracranial generators estimated in the posterior cingulate cortex. These new findings suggest the existence of dissociable brain systems depending on feedback valence and expectancy. More generally, they inform about the added value of using topographic ERP mapping methods, besides conventional ERP measurements, to characterize qualitative changes occurring in the spatio-temporal dynamic of reward processing during PM. Copyright © 2017 Elsevier B.V. All rights reserved.
-
The Role of the Rat Medial Prefrontal Cortex in Adapting to Changes in Instrumental Contingency
Coutureau, Etienne; Esclassan, Frederic; Di Scala, Georges; Marchand, Alain R.
2012-01-01
In order to select actions appropriate to current needs, a subject must identify relationships between actions and events. Control over the environment is determined by the degree to which action consequences can be predicted, as described by action-outcome contingencies – i.e. performing an action should affect the probability of the outcome. We evaluated in a first experiment adaptation to contingency changes in rats with neurotoxic lesions of the medial prefrontal cortex. Results indicate that this brain region is not critical to adjust instrumental responding to a negative contingency where the rats must refrain from pressing a lever, as this action prevents reward delivery. By contrast, this brain region is required to reduce responding in a non-contingent situation where the same number of rewards is freely delivered and actions do not affect the outcome any more. In a second experiment, we determined that this effect does not result from a different perception of temporal relationships between actions and outcomes since lesioned rats adapted normally to gradually increasing delays in reward delivery. These data indicate that the medial prefrontal cortex is not directly involved in evaluating the correlation between action-and reward-rates or in the perception of reward delays. The deficit in lesioned rats appears to consist of an abnormal response to the balance between contingent and non-contingent rewards. By highlighting the role of prefrontal regions in adapting to the causal status of actions, these data contribute to our understanding of the neural basis of choice tasks. PMID:22496747
-
A Representation of Effort in Decision-Making and Motor Control.
Shadmehr, Reza; Huang, Helen J; Ahmed, Alaa A
2016-07-25
Given two rewarding stimuli, animals tend to choose the more rewarding (or less effortful) option. However, they also move faster toward that stimulus [1-5]. This suggests that reward and effort not only affect decision-making, they also influence motor control [6, 7]. How does the brain compute the effort requirements of a task? Here, we considered data acquired during walking, reaching, flying, or isometric force production. In analyzing the decision-making and motor-control behaviors of various animals, we considered the possibility that the brain may estimate effort objectively, via the metabolic energy consumed to produce the action. We measured the energetic cost of reaching and found that, like walking, it was convex in time, with a global minimum, implying that there existed a movement speed that minimized effort. However, reward made it worthwhile to be energetically inefficient. Using a framework in which utility of an action depended on reward and energetic cost, both discounted in time, we found that it was possible to account for a body of data in which animals were free to choose how to move (reach slow or fast), as well as what to do (walk or fly, produce force F1 or F2). We suggest that some forms of decision-making and motor control may share a common utility in which the brain represents the effort associated with performing an action objectively via its metabolic energy cost and then, like reward, temporally discounts it as a function of movement duration. Copyright © 2016 Elsevier Ltd. All rights reserved.
-
ERIC Educational Resources Information Center
Urosevic, Snezana; Collins, Paul; Muetzel, Ryan; Lim, Kelvin; Luciana, Monica
2012-01-01
Adolescence is a period of radical normative changes and increased risk for substance use, mood disorders, and physical injury. Researchers have proposed that increases in reward sensitivity (i.e., sensitivity of the behavioral approach system [BAS]) and/or increases in reactivity to all emotional stimuli (i.e., reward and threat sensitivities)…
-
Letting go of the present: mind-wandering is associated with reduced delay discounting.
Smallwood, Jonathan; Ruby, Florence J M; Singer, Tania
2013-03-01
The capacity to self-generate mental content that is unrelated to the current environment is a fundamental characteristic of the mind, and the current experiment explored how this experience is related to the decisions that people make in daily life. We examined how task-unrelated thought (TUT) varies with the length of time participants are willing to wait for an economic reward, as measured using an inter-temporal discounting task. When participants performed a task requiring minimal attention, the greater the amount of time spent engaged in TUT the longer the individual was prepared to wait for an economic reward. These data indicate that self-generated thought engages processes associated with the successful management of long-term goals. Although immersion in the here and now is undeniably advantageous, under appropriate conditions the capacity to let go of the present and consider more pertinent personal goals may have its own rewards. Copyright © 2012 Elsevier Inc. All rights reserved.
-
Blancas, A; González-García, S D; Rodríguez, K; Escobar, C
2014-12-05
Scheduled and restricted access to a palatable snack, i.e. chocolate, elicits a brief and strong anticipatory activation and entrains brain areas related with reward and motivation. This behavioral and neuronal activation persists for more than 7days when this protocol is interrupted, suggesting the participation of a time-keeping system. The process that initiates this anticipation may provide a further understanding of the time-keeping system underlying palatable food entrainment. The aim of this study was to analyze how this entraining protocol starts and to dissect neuronal structures that initiate a chocolate-entrained activation. We assessed the development of anticipation of 5g of chocolate during the first 8days of the entrainment protocol. General activity of control and chocolate-entrained rats was continuously monitored with movement sensors. Moreover, motivation to obtain the chocolate was assessed by measuring approaches and interaction responses toward a wire-mesh box containing chocolate. Neuronal activation was determined with c-Fos in reward-related brain areas. We report a progressive increase in the interaction with a box to obtain chocolate parallel to a progressive neuronal activation. A significant anticipatory activation was observed in the prefrontal cortex on day 3 of entrainment and in the nucleus accumbens on day 5, while the arcuate nucleus and pyriform cortex reached significant activation on day 8. The gradual response observed with this protocol indicates that anticipation of a rewarding food requires repetitive and predictable experiences in order to acquire a temporal estimation. We also confirm that anticipation of palatable food involves diverse brain regions. Copyright © 2014 IBRO. Published by Elsevier Ltd. All rights reserved.
-
Samanez-Larkin, Gregory R; Levens, Sara M; Perry, Lee M; Dougherty, Robert F; Knutson, Brian
2012-04-11
Frontostriatal circuits have been implicated in reward learning, and emerging findings suggest that frontal white matter structural integrity and probabilistic reward learning are reduced in older age. This cross-sectional study examined whether age differences in frontostriatal white matter integrity could account for age differences in reward learning in a community life span sample of human adults. By combining diffusion tensor imaging with a probabilistic reward learning task, we found that older age was associated with decreased reward learning and decreased white matter integrity in specific pathways running from the thalamus to the medial prefrontal cortex and from the medial prefrontal cortex to the ventral striatum. Further, white matter integrity in these thalamocorticostriatal paths could statistically account for age differences in learning. These findings suggest that the integrity of frontostriatal white matter pathways critically supports reward learning. The findings also raise the possibility that interventions that bolster frontostriatal integrity might improve reward learning and decision making.
-
Moreno-López, Laura; Soriano-Mas, Carles; Delgado-Rico, Elena; Rio-Valle, Jacqueline S; Verdejo-García, Antonio
2012-01-01
Neuroscience evidence suggests that adolescent obesity is linked to brain dysfunctions associated with enhanced reward and somatosensory processing and reduced impulse control during food processing. Comparatively less is known about the role of more stable brain structural measures and their link to personality traits and neuropsychological factors on the presentation of adolescent obesity. Here we aimed to investigate regional brain anatomy in adolescents with excess weight vs. lean controls. We also aimed to contrast the associations between brain structure and personality and cognitive measures in both groups. Fifty-two adolescents (16 with normal weight and 36 with excess weight) were scanned using magnetic resonance imaging and completed the Sensitivity to Punishment and Sensitivity to Reward Questionnaire (SPSRQ), the UPPS-P scale, and the Stroop task. Voxel-based morphometry (VBM) was used to assess possible between-group differences in regional gray matter (GM) and to measure the putative differences in the way reward and punishment sensitivity, impulsivity and inhibitory control relate to regional GM volumes, which were analyzed using both region of interest (ROI) and whole brain analyses. The ROIs included areas involved in reward/somatosensory processing (striatum, somatosensory cortices) and motivation/impulse control (hippocampus, prefrontal cortex). Excess weight adolescents showed increased GM volume in the right hippocampus. Voxel-wise volumes of the second somatosensory cortex (SII) were correlated with reward sensitivity and positive urgency in lean controls, but this association was missed in excess weight adolescents. Moreover, Stroop performance correlated with dorsolateral prefrontal cortex volumes in controls but not in excess weight adolescents. Adolescents with excess weight have structural abnormalities in brain regions associated with somatosensory processing and motivation.
-
Seitovirta, Jaana; Vehviläinen-Julkunen, Katri; Mitronen, Lasse; De Gieter, Sara; Kvist, Tarja
2017-04-01
To identify meaningful types of rewards and the consequences of rewards as expressed by Finnish registered nurses working in primary and private healthcare. Previous studies have found significant associations between nurses' rewards and both their commitment and job satisfaction. Furthermore, appropriate rewards can have beneficial effects on factors including workforce stability and occupational satisfaction that are highly important in times of nurse shortages. A cross-sectional, qualitative interview study. Data were collected via individual semi-structured interviews (n = 20) with registered nurses working in Finland's primary and private healthcare, and subjected to qualitative content analysis. Six meaningful types of rewards were identified by the registered nurses: Financial compensation and benefits, Work-Life balance, Work content, Professional development, Recognition, and Supportive leadership. Rewards encouraged respondents to perform their work correctly and reinforced occupational satisfaction, but also caused feelings of envy and stress. It is essential to pay attention to nurses' preferences for particular rewards and to reward management. When designing effective reward systems for registered nurses, it is not sufficient to provide financial rewards alone, as various kinds of non-financial rewards are both meaningful and necessary. When trying to improve registered nurses' commitment and job satisfaction through reward management, it is important to listen to nurses' opinions to create a reward system that integrates financial and non-financial rewards and is fair from their perspective. Healthcare organisations that offer registered nurses a holistic reward system are more likely to retain satisfied and committed nurses at a time of increasing nursing shortages. © 2016 John Wiley & Sons Ltd.
-
A Simple Network Architecture Accounts for Diverse Reward Time Responses in Primary Visual Cortex
Hussain Shuler, Marshall G.; Shouval, Harel Z.
2015-01-01
Many actions performed by animals and humans depend on an ability to learn, estimate, and produce temporal intervals of behavioral relevance. Exemplifying such learning of cued expectancies is the observation of reward-timing activity in the primary visual cortex (V1) of rodents, wherein neural responses to visual cues come to predict the time of future reward as behaviorally experienced in the past. These reward-timing responses exhibit significant heterogeneity in at least three qualitatively distinct classes: sustained increase or sustained decrease in firing rate until the time of expected reward, and a class of cells that reach a peak in firing at the expected delay. We elaborate upon our existing model by including inhibitory and excitatory units while imposing simple connectivity rules to demonstrate what role these inhibitory elements and the simple architectures play in sculpting the response dynamics of the network. We find that simply adding inhibition is not sufficient for obtaining the different distinct response classes, and that a broad distribution of inhibitory projections is necessary for obtaining peak-type responses. Furthermore, although changes in connection strength that modulate the effects of inhibition onto excitatory units have a strong impact on the firing rate profile of these peaked responses, the network exhibits robustness in its overall ability to predict the expected time of reward. Finally, we demonstrate how the magnitude of expected reward can be encoded at the expected delay in the network and how peaked responses express this reward expectancy. SIGNIFICANCE STATEMENT Heterogeneity in single-neuron responses is a common feature of neuronal systems, although sometimes, in theoretical approaches, it is treated as a nuisance and seldom considered as conveying a different aspect of a signal. In this study, we focus on the heterogeneous responses in the primary visual cortex of rodents trained with a predictable delayed reward time. We describe under what conditions this heterogeneity can arise by self-organization, and what information it can convey. This study, while focusing on a specific system, provides insight onto how heterogeneity can arise in general while also shedding light onto mechanisms of reinforcement learning using realistic biological assumptions. PMID:26377457
-
Differential Encoding of Time by Prefrontal and Striatal Network Dynamics.
Bakhurin, Konstantin I; Goudar, Vishwa; Shobe, Justin L; Claar, Leslie D; Buonomano, Dean V; Masmanidis, Sotiris C
2017-01-25
Telling time is fundamental to many forms of learning and behavior, including the anticipation of rewarding events. Although the neural mechanisms underlying timing remain unknown, computational models have proposed that the brain represents time in the dynamics of neural networks. Consistent with this hypothesis, changing patterns of neural activity dynamically in a number of brain areas-including the striatum and cortex-has been shown to encode elapsed time. To date, however, no studies have explicitly quantified and contrasted how well different areas encode time by recording large numbers of units simultaneously from more than one area. Here, we performed large-scale extracellular recordings in the striatum and orbitofrontal cortex of mice that learned the temporal relationship between a stimulus and a reward and reported their response with anticipatory licking. We used a machine-learning algorithm to quantify how well populations of neurons encoded elapsed time from stimulus onset. Both the striatal and cortical networks encoded time, but the striatal network outperformed the orbitofrontal cortex, a finding replicated both in simultaneously and nonsimultaneously recorded corticostriatal datasets. The striatal network was also more reliable in predicting when the animals would lick up to ∼1 s before the actual lick occurred. Our results are consistent with the hypothesis that temporal information is encoded in a widely distributed manner throughout multiple brain areas, but that the striatum may have a privileged role in timing because it has a more accurate "clock" as it integrates information across multiple cortical areas. The neural representation of time is thought to be distributed across multiple functionally specialized brain structures, including the striatum and cortex. However, until now, the neural code for time has not been compared quantitatively between these areas. Here, we performed large-scale recordings in the striatum and orbitofrontal cortex of mice trained on a stimulus-reward association task involving a delay period and used a machine-learning algorithm to quantify how well populations of simultaneously recorded neurons encoded elapsed time from stimulus onset. We found that, although both areas encoded time, the striatum consistently outperformed the orbitofrontal cortex. These results suggest that the striatum may refine the code for time by integrating information from multiple inputs. Copyright © 2017 the authors 0270-6474/17/370854-17$15.00/0.
-
Previous Cocaine Exposure Makes Rats Hypersensitive to Both Delay and Reward Magnitude
Roesch, Matthew R.; Takahashi, Yuji; Gugsa, Nishan; Bissonette, Gregory B.; Schoenbaum, Geoffrey
2008-01-01
Animals prefer an immediate over a delayed reward, just as they prefer a large over a small reward. Exposure to psychostimulants causes long-lasting changes in structures critical for this behavior and might disrupt normal time-discounting performance. To test this hypothesis, we exposed rats to cocaine daily for 2 weeks (30 mg/kg, i.p.). Approximately 6 weeks later, we tested them on a variant of a time-discounting task, in which the rats responded to one of two locations to obtain reward while we independently manipulated the delay to reward and reward magnitude. Performance did not differ between cocaine-treated and saline-treated (control) rats when delay lengths and reward magnitudes were equal at the two locations. However, cocaine-treated rats were significantly more likely to shift their responding when we increased the delay or reward size asymmetrically. Furthermore, they were slower to respond and made more errors when forced to the side associated with the lower value. We conclude that previous exposure to cocaine makes choice behavior hypersensitive to differences in the time to and size of available rewards, consistent with a general effect of cocaine exposure on reward valuation mechanisms. PMID:17202492
-
Previous cocaine exposure makes rats hypersensitive to both delay and reward magnitude.
Roesch, Matthew R; Takahashi, Yuji; Gugsa, Nishan; Bissonette, Gregory B; Schoenbaum, Geoffrey
2007-01-03
Animals prefer an immediate over a delayed reward, just as they prefer a large over a small reward. Exposure to psychostimulants causes long-lasting changes in structures critical for this behavior and might disrupt normal time-discounting performance. To test this hypothesis, we exposed rats to cocaine daily for 2 weeks (30 mg/kg, i.p.). Approximately 6 weeks later, we tested them on a variant of a time-discounting task, in which the rats responded to one of two locations to obtain reward while we independently manipulated the delay to reward and reward magnitude. Performance did not differ between cocaine-treated and saline-treated (control) rats when delay lengths and reward magnitudes were equal at the two locations. However, cocaine-treated rats were significantly more likely to shift their responding when we increased the delay or reward size asymmetrically. Furthermore, they were slower to respond and made more errors when forced to the side associated with the lower value. We conclude that previous exposure to cocaine makes choice behavior hypersensitive to differences in the time to and size of available rewards, consistent with a general effect of cocaine exposure on reward valuation mechanisms.
-
Reward for food odors: an fMRI study of liking and wanting as a function of metabolic state and BMI
Jiang, Tao; Soussignan, Robert; Schaal, Benoist
2015-01-01
Brain reward systems mediate liking and wanting for food reward. Here, we explore the differential involvement of the following structures for these two components: the ventral and dorsal striatopallidal area, orbitofrontal cortex (OFC), anterior insula and anterior cingulate. Twelve healthy female participants were asked to rate pleasantness (liking of food and non-food odors) and the desire to eat (wanting of odor-evoked food) during event-related functional magnetic resonance imaging (fMRI). The subjective ratings and fMRI were performed in hunger and satiety states. Activations of regions of interest were compared as a function of task (liking vs wanting), odor category (food vs non-food) and metabolic state (hunger vs satiety). We found that the nucleus accumbens and ventral pallidum were differentially involved in liking or wanting during the hunger state, which suggests a reciprocal inhibitory influence between these structures. Neural activation of OFC subregions was correlated with either liking or wanting ratings, suggesting an OFC role in reward processing magnitude. Finally, during the hunger state, participants with a high body mass index exhibited less activation in neural structures underlying food reward processing. Our results suggest that food liking and wanting are two separable psychological constructs and may be functionally segregated within the cortico-striatopallidal circuit. PMID:24948157
-
The Brain Reward Circuitry in Mood Disorders
Russo, Scott J.; Nestler, Eric J.
2013-01-01
Mood disorders are common and debilitating conditions characterized in part by profound deficits in reward-related behavioral domains. A recent literature has identified important structural and functional alterations within the brain’s reward circuitry —particularly in the ventral tegmental area to nucleus accumbens pathway — that are associated with symptoms such as anhedonia and aberrant reward-associated perception and memory. This review synthesizes recent data from human and rodent studies from which emerges a circuit-level framework for understanding reward deficits in depression. We also discuss some of the molecular and cellular underpinnings of this framework, ranging from adaptations in glutamatergic synapses and neurotrophic factors to transcriptional and epigenetic mechanisms. PMID:23942470
-
Reward skewness coding in the insula independent of probability and loss
Tobler, Philippe N.
2011-01-01
Rewards in the natural environment are rarely predicted with complete certainty. Uncertainty relating to future rewards has typically been defined as the variance of the potential outcomes. However, the asymmetry of predicted reward distributions, known as skewness, constitutes a distinct but neuroscientifically underexplored risk term that may also have an impact on preference. By changing only reward magnitudes, we study skewness processing in equiprobable ternary lotteries involving only gains and constant probabilities, thus excluding probability distortion or loss aversion as mechanisms for skewness preference formation. We show that individual preferences are sensitive to not only the mean and variance but also to the skewness of predicted reward distributions. Using neuroimaging, we show that the insula, a structure previously implicated in the processing of reward-related uncertainty, responds to the skewness of predicted reward distributions. Some insula responses increased in a monotonic fashion with skewness (irrespective of individual skewness preferences), whereas others were similarly elevated to both negative and positive as opposed to no reward skew. These data support the notion that the asymmetry of reward distributions is processed in the brain and, taken together with replicated findings of mean coding in the striatum and variance coding in the cingulate, suggest that the brain codes distinct aspects of reward distributions in a distributed fashion. PMID:21849610
-
Anatomical correlates of reward-seeking behaviours in behavioural variant frontotemporal dementia
Sturm, Virginia E.; Seeley, William W.; Miller, Bruce L.; Kramer, Joel H.; Rosen, Howard J.
2014-01-01
Behavioural variant frontotemporal dementia is characterized by abnormal responses to primary reward stimuli such as food, sex and intoxicants, suggesting abnormal functioning of brain circuitry mediating reward processing. The goal of this analysis was to determine whether abnormalities in reward-seeking behaviour in behavioural variant frontotemporal dementia are correlated with atrophy in regions known to mediate reward processing. Review of case histories in 103 patients with behavioural variant frontotemporal dementia identified overeating or increased sweet food preference in 80 (78%), new or increased alcohol or drug use in 27 (26%), and hypersexuality in 17 (17%). For each patient, a primary reward-seeking score of 0–3 was created with 1 point given for each target behaviour (increased seeking of food, drugs, or sex). Voxel-based morphometry performed in 91 patients with available imaging revealed that right ventral putamen and pallidum atrophy correlated with higher reward-seeking scores. Each of the reward-related behaviours involved partially overlapping right hemisphere reward circuit regions including putamen, globus pallidus, insula and thalamus. These findings indicate that in some patients with behavioural variant frontotemporal dementia, low volume of subcortical reward-related structures is associated with increased pursuit of primary rewards, which may be a product of increased thalamocortical feedback. PMID:24740987
-
Wakabayashi, Ken T; Bruno, Michael J; Bass, Caroline E; Park, Jinwoo
2016-06-21
The olfactory tubercle (OT), as a component of the ventral striatum, serves as an important multisensory integration center for reward-related processes in the brain. Recent studies show that dense dopaminergic innervation from the ventral tegmental area (VTA) into the OT may play an outsized role in disorders such as psychostimulant addiction and disorders of motivation, increasing recent scientific interest in this brain region. However, due to its anatomical inaccessibility, relative small size, and proximity to other dopamine-rich structures, neurochemical assessments using conventional methods cannot be readily employed. Here, we investigated dopamine (DA) regulation in the OT of urethane-anesthetized rats using in vivo fast-scan voltammetry (FSCV) coupled with carbon-fiber microelectrodes, following optogenetic stimulation of the VTA. The results were compared with DA regulation in the nucleus accumbens (NAc), a structure located adjacent to the OT and which also receives dense DA innervation from the VTA. FSCV coupled with optically evoked release allowed us to investigate the spatial distribution of DA in the OT and characterize OT DA dynamics (release and clearance) with subsecond temporal and micrometer spatial resolution for the first time. In this study, we demonstrated that DA transporters play an important role in regulating DA in the OT. However, the control of extracellular DA by uptake in the OT was less than in the NAc. The difference in DA transmission in the terminal fields of the OT and NAc may be involved in region-specific responses to drugs of abuse and contrasting roles in mediating reward-related behavior.
-
Use of reward-penalty structures in human experimentation
NASA Technical Reports Server (NTRS)
Stein, A. C.; Allen, R. W.; Schwartz, S. H.
1978-01-01
The use of motivational techniques in human performance research is reviewed and an example study employing a reward-penalty structure to simulate the motivations inherent in a real-world situation is presented. Driver behavior in a decision-making driving scenario was studied. The task involved control of an instrumented car on a cooperative test course. Subjects were penalized monetarily for tickets and accidents and rewarded for saving driving time. Two groups were assigned different ticket penalties. The group with the highest penalties tended to drive more conservatively. However, the average total payoff to each group was the same, as the conservative drivers traded off slower driving times with lower ticket penalties.
-
Simon, Joe J; Skunde, Mandy; Wu, Mudan; Schnell, Knut; Herpertz, Sabine C; Bendszus, Martin; Herzog, Wolfgang; Friederich, Hans-Christoph
2015-08-01
Food is an innate reward stimulus related to energy homeostasis and survival, whereas money is considered a more general reward stimulus that gains a rewarding value through learning experiences. Although the underlying neural processing for both modalities of reward has been investigated independently from one another, a more detailed investigation of neural similarities and/or differences between food and monetary reward is still missing. Here, we investigated the neural processing of food compared with monetary-related rewards in 27 healthy, normal-weight women using functional magnetic resonance imaging. We developed a task distinguishing between the anticipation and the receipt of either abstract food or monetary reward. Both tasks activated the ventral striatum during the expectation of a reward. Compared with money, greater food-related activations were observed in prefrontal, parietal and central midline structures during the anticipation and lateral orbitofrontal cortex (lOFC) during the receipt of food reward. Furthermore, during the receipt of food reward, brain activation in the secondary taste cortex was positively related to the body mass index. These results indicate that food-dependent activations encompass to a greater extent brain regions involved in self-control and self-reflection during the anticipation and phylogenetically older parts of the lOFC during the receipt of reward. © The Author (2014). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
Mehta, Mitul A; Gore-Langton, Emma; Golembo, Nicole; Colvert, Emma; Williams, Steven C R; Sonuga-Barke, Edmund
2010-10-01
Severe deprivation in the first few years of life is associated with multiple difficulties in cognition and behavior. However, the brain basis for these difficulties is poorly understood. Structural and functional neuroimaging studies have implicated limbic system structures as dysfunctional, and one functional imaging study in a heterogeneous group of maltreated individuals has confirmed the presence of abnormalities in the basal ganglia. Based on these studies and known dopaminergic abnormalities from studies in experimental animals using social isolation, we used a task of monetary reward anticipation to examine the functional integrity of brain regions previously shown to be implicated in reward processing. Our sample included a group of adolescents (n = 12) who had experienced global deprivation early in their lives in Romania prior to adoption into UK families. In contrast to a nonadopted comparison group (n = 11), the adoptees did not recruit the striatum during reward anticipation despite comparable performance accuracy and latency. These results show, for the first time, an association between early institutional deprivation and brain reward systems in humans and highlight potential neural vulnerabilities resulting from such exposures.
-
The Rewarding Aspects of Music Listening Are Related to Degree of Emotional Arousal
Salimpoor, Valorie N.; Benovoy, Mitchel; Longo, Gregory; Cooperstock, Jeremy R.; Zatorre, Robert J.
2009-01-01
Background Listening to music is amongst the most rewarding experiences for humans. Music has no functional resemblance to other rewarding stimuli, and has no demonstrated biological value, yet individuals continue listening to music for pleasure. It has been suggested that the pleasurable aspects of music listening are related to a change in emotional arousal, although this link has not been directly investigated. In this study, using methods of high temporal sensitivity we investigated whether there is a systematic relationship between dynamic increases in pleasure states and physiological indicators of emotional arousal, including changes in heart rate, respiration, electrodermal activity, body temperature, and blood volume pulse. Methodology Twenty-six participants listened to self-selected intensely pleasurable music and “neutral” music that was individually selected for them based on low pleasure ratings they provided on other participants' music. The “chills” phenomenon was used to index intensely pleasurable responses to music. During music listening, continuous real-time recordings of subjective pleasure states and simultaneous recordings of sympathetic nervous system activity, an objective measure of emotional arousal, were obtained. Principal Findings Results revealed a strong positive correlation between ratings of pleasure and emotional arousal. Importantly, a dissociation was revealed as individuals who did not experience pleasure also showed no significant increases in emotional arousal. Conclusions/Significance These results have broader implications by demonstrating that strongly felt emotions could be rewarding in themselves in the absence of a physically tangible reward or a specific functional goal. PMID:19834599
-
Activity of striatal neurons reflects social action and own reward.
Báez-Mendoza, Raymundo; Harris, Christopher J; Schultz, Wolfram
2013-10-08
Social interactions provide agents with the opportunity to earn higher benefits than when acting alone and contribute to evolutionary stable strategies. A basic requirement for engaging in beneficial social interactions is to recognize the actor whose movement results in reward. Despite the recent interest in the neural basis of social interactions, the neurophysiological mechanisms identifying the actor in social reward situations are unknown. A brain structure well suited for exploring this issue is the striatum, which plays a role in movement, reward, and goal-directed behavior. In humans, the striatum is involved in social processes related to reward inequity, donations to charity, and observational learning. We studied the neurophysiology of social action for reward in rhesus monkeys performing a reward-giving task. The behavioral data showed that the animals distinguished between their own and the conspecific's reward and knew which individual acted. Striatal neurons coded primarily own reward but rarely other's reward. Importantly, the activations occurred preferentially, and in approximately similar fractions, when either the own or the conspecific's action was followed by own reward. Other striatal neurons showed social action coding without reward. Some of the social action coding disappeared when the conspecific's role was simulated by a computer, confirming a social rather than observational relationship. These findings demonstrate a role of striatal neurons in identifying the social actor and own reward in a social setting. These processes may provide basic building blocks underlying the brain's function in social interactions.
-
Identifying nurses' rewards: a qualitative categorization study in Belgium
De Gieter, Sara; De Cooman, Rein; Pepermans, Roland; Caers, Ralf; Du Bois, Cindy; Jegers, Marc
2006-01-01
Background Rewards are important in attracting, motivating and retaining the most qualified employees, and nurses are no exception to this rule. This makes the establishment of an efficient reward system for nurses a true challenge for every hospital manager. A reward does not necessarily have a financial connotation: non-financial rewards may matter too, or may even be more important. Therefore, the present study examines nurses' reward perceptions, in order to identify potential reward options. Methods To answer the research question "What do nurses consider a reward and how can these rewards be categorized?", 20 in-depth semi-structured interviews with nurses were conducted and analysed using discourse and content analyses. In addition, the respondents received a list of 34 rewards (derived from the literature) and were asked to indicate the extent to which they perceived each of them to be rewarding. Results Discourse analysis revealed three major reward categories: financial, non-financial and psychological, each containing different subcategories. In general, nurses more often mentioned financial rewards spontaneously in the interview, compared to non-financial and psychological rewards. The questionnaire results did not, however, indicate a significant difference in the rewarding potential of these three categories. Both the qualitative and quantitative data revealed that a number of psychological and non-financial rewards were important for nurses in addition to their monthly pay and other remunerations. In particular, appreciation for their work by others, compliments from others, presents from others and contact with patients were highly valued. Moreover, some demographical variables influenced the reward perceptions. Younger and less experienced nurses considered promotion possibilities as more rewarding than the older and more senior ones. The latter valued job security and working for a hospital with a good reputation higher than their younger and more junior colleagues. Conclusion When trying to establish an efficient reward system for nurses, hospital managers should not concentrate on the financial reward possibilities alone. They also ought to consider non-financial and psychological rewards (in combination with financial rewards), since nurses value these as well and they may lead to a more personalized reward system. PMID:16824227
-
Namboodiri, Vijay Mohan K.; Levy, Joshua M.; Mihalas, Stefan; Sims, David W.; Hussain Shuler, Marshall G.
2016-01-01
Understanding the exploration patterns of foragers in the wild provides fundamental insight into animal behavior. Recent experimental evidence has demonstrated that path lengths (distances between consecutive turns) taken by foragers are well fitted by a power law distribution. Numerous theoretical contributions have posited that “Lévy random walks”—which can produce power law path length distributions—are optimal for memoryless agents searching a sparse reward landscape. It is unclear, however, whether such a strategy is efficient for cognitively complex agents, from wild animals to humans. Here, we developed a model to explain the emergence of apparent power law path length distributions in animals that can learn about their environments. In our model, the agent’s goal during search is to build an internal model of the distribution of rewards in space that takes into account the cost of time to reach distant locations (i.e., temporally discounting rewards). For an agent with such a goal, we find that an optimal model of exploration in fact produces hyperbolic path lengths, which are well approximated by power laws. We then provide support for our model by showing that humans in a laboratory spatial exploration task search space systematically and modify their search patterns under a cost of time. In addition, we find that path length distributions in a large dataset obtained from free-ranging marine vertebrates are well described by our hyperbolic model. Thus, we provide a general theoretical framework for understanding spatial exploration patterns of cognitively complex foragers. PMID:27385831
-
Cansell, Céline; Castel, Julien; Denis, Raphaël G. P.; Rouch, Claude; Delbes, Anne-Sophie; Martinez, Sarah; Mestivier, Denis; Finan, Brian; Maldonado-Aviles, Jaime G.; Rijnsburger, Merel; Tschöp, Matthias H.; DiLeone, Ralph J.; Eckel, Robert H.; la Fleur, Susanne E.; Magnan, Christophe; Hnasko, Thomas S.; Luquet, Serge
2014-01-01
Circulating triglycerides (TG) normally increase after a meal but are altered in pathophysiological conditions such as obesity. Although TG metabolism in the brain remains poorly understood, several brain structures express enzymes that process TG-enriched particles, including mesolimbic structures. For this reason, and because consumption of high fat diet alters dopamine signaling, we tested the hypothesis that TG might directly target mesolimbic reward circuits to control reward-seeking behaviors. We found that the delivery of small amounts of TG to the brain through the carotid artery rapidly reduced both spontaneous and amphetamine-induced locomotion, abolished preference for palatable food, and reduced the motivation to engage in food-seeking behavior. Conversely, targeted disruption of the TG-hydrolyzing enzyme lipoprotein lipase specifically in the nucleus accumbens increased palatable food preference and food seeking behavior. Finally, prolonged TG perfusion resulted in a return to normal palatable food preference despite continued locomotor suppression, suggesting that adaptive mechanisms occur. These findings reveal new mechanisms by which dietary fat may alter mesolimbic circuit function and reward seeking. PMID:24732670
-
Brain activity and infant attachment history in young men during loss and reward processing.
Quevedo, Karina; Waters, Theodore E A; Scott, Hannah; Roisman, Glenn I; Shaw, Daniel S; Forbes, Erika E
2017-05-01
There is now ample evidence that the quality of early attachment experiences shapes expectations for supportive and responsive care and ultimately serves to scaffold adaptation to the salient tasks of development. Nonetheless, few studies have identified neural mechanisms that might give rise to these associations. Using a moderately large sample of low-income male participants recruited during infancy (N = 171), we studied the predictive significance of attachment insecurity and disorganization at age 18 months (as measured in the Strange Situation Procedure) for patterns of neural activation to reward and loss at age 20 years (assessed during a reward-based task as part of a functional magnetic resonance imaging scan). Results indicated that individuals with a history of insecure attachment showed hyperactivity in (a) reward- and emotion-related (e.g., basal ganglia and amygdala) structures and (b) emotion regulation and self-referential processing (cortical midline structures) in response to positive and negative outcomes (and anticipation of those outcomes). Further, the neural activation of individuals with a history of disorganized attachment suggested that they had greater emotional reactivity in anticipation of reward and employed greater cognitive control when negative outcomes were encountered. Overall, results suggest that the quality of early attachments has lasting impacts on brain function and reward processing.
-
fMRI study of neural sensitization to hedonic stimuli in long‐term, daily cannabis users
Dunlop, Joseph; Ketcherside, Ariel; Baine, Jessica; Rhinehardt, Tyler; Kuhn, Brittany; DeWitt, Sam; Alvi, Talha
2016-01-01
Abstract Although there is emergent evidence illustrating neural sensitivity to cannabis cues in cannabis users, the specificity of this effect to cannabis cues as opposed to a generalized hyper‐sensitivity to hedonic stimuli has not yet been directly tested. Using fMRI, we presented 53 daily, long‐term cannabis users and 68 non‐using controls visual and tactile cues for cannabis, a natural reward, and, a sensory‐perceptual control object to evaluate brain response to hedonic stimuli in cannabis users. The results showed an interaction between group and reward type such that the users had greater response during cannabis cues relative to natural reward cues (i.e., fruit) in the orbitofrontal cortex, striatum, anterior cingulate gyrus, and ventral tegmental area compared to non‐users (cluster‐threshold z = 2.3, P < 0.05). In the users, there were positive brain‐behavior correlations between neural response to cannabis cues in fronto‐striatal‐temporal regions and subjective craving, marijuana‐related problems, withdrawal symptoms, and levels of THC metabolites (cluster‐threshold z = 2.3, P < 0.05). These findings demonstrate hyper‐responsivity, and, specificity of brain response to cannabis cues in long‐term cannabis users that are above that of response to natural reward cues. These observations are concordant with incentive sensitization models suggesting sensitization of mesocorticolimbic regions and disruption of natural reward processes following drug use. Although the cross‐sectional nature of this study does not provide information on causality, the positive correlations between neural response and indicators of cannabis use (i.e., THC levels) suggest that alterations in the reward system are, in part, related to cannabis use. Hum Brain Mapp 37:3431–3443, 2016. © 2016 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc. PMID:27168331
-
Toossi, Saied
2017-10-01
Although previous studies have established the effectiveness of using small reward-based incentives in inducing the choice and consumption of healthier foods among children, little is known about their impact outside of experimental settings or their effectiveness over time when administered daily. This paper presents the results of a field experiment conducted to provide insight on these matters. The study employs a removed treatment within-subject design and was conducted at a summer program catering to low-income children between the ages of 5 and 12. The month long experiment-wherein participants were offered a small prize for choosing a fruit cup for dessert after lunch in lieu of cookies-involved 23 children between the ages of 5 and 8. Corroborating existing studies, the introduction of small reward-based incentives in this context was found to induce large increases in the number of children choosing the healthy dessert options after lunch, but disaggregating the results by week and day suggests that their impact diminished over time. Attempts to ascertain their effect outside of experimental settings did not indicate that the introduction of rewards had any adverse effects, but also did not provide definitive conclusions. Copyright © 2017 Elsevier Ltd. All rights reserved.
-
Effect of Adolescent Bariatric Surgery on the Brain and Cognition: A Pilot Study.
Pearce, Alaina L; Mackey, Eleanor; Cherry, J Bradley C; Olson, Alexandra; You, Xiaozhen; Magge, Sheela N; Mietus-Snyder, Michele; Nadler, Evan P; Vaidya, Chandan J
2017-11-01
Neurocognitive deficits in pediatric obesity relate to poor developmental outcomes. We sought preliminary evidence for changes in brain and cognitive functioning relevant to obesogenic behavior following vertical sleeve gastrectomy (VSG) in adolescents relative to wait-listed (WL) and healthy controls (HC). Thirty-six adolescents underwent fMRI twice 4 months apart, during executive, reward, and episodic memory encoding, in addition to behavioral testing for reward-related decision making. VSG adolescents lost weight, while WL gained weight and HC did not change between time points. Gains in executive and reward-related performance were larger in VSG than control groups. Group × Time interaction (P < 0.05 corrected) in left prefrontal cortex during N-back showed greater presurgical activation and postsurgical reduction comparable to HC levels but increased in WL between time points. Similarly, left striatal parametric response to reward value reduced after surgery to HC levels; WL did not change. Memory-related medial temporal activation did not change in any group. Results provide pilot evidence for functional brain changes induced by VSG in adolescents with severe obesity. Weight loss and gain were paralleled by reduced and increased prefrontal activation, respectively, suggesting neural plasticity related to metabolic change. © 2017 The Obesity Society.
-
Wu, Haiyan; Gui, Danyang; Lin, Wenzheng; Gu, Ruolei; Zhu, Xiangru; Liu, Xun
2016-08-01
Much past research has focused on the correlation between procrastination and personality traits (e.g., impulsivity). According to the temporal motivation theory, procrastinators are impulsive and sensitive to delays in time. However, there is still a lack of direct evidence of the tendency of procrastinators to prefer immediate over future rewards. To investigate this question, we recorded event-related potentials (ERPs) in the brain while participants performed an intertemporal choice task involving both time delay and reward processing. The participants were assigned to a high procrastination group and a low procrastination group according to their scores on self-report measures. We found that high procrastination participants preferred immediate rewards compared to future ones whereas low procrastination participants did not. High procrastinators also exhibited a larger and delayed P2 component, indicating delay time processing and abnormal reward processing. No significant effect associated with procrastination was found on the P300 component. Taken together, these findings suggest that high procrastinators are more impulsive and encode the information of delay time more slowly but with a higher level of motivation-driven attention. The current study substantiates higher impulsivity in procrastination and verifies that a difference exists in the sensitivity to time delay between high and low procrastinators. Copyright © 2016 Elsevier Inc. All rights reserved.
-
Fulford, Daniel; Campellone, Tim; Gard, David E
2018-05-28
Limited quantity and quality of interpersonal exchanges and relationships predict worse symptomatic and hospitalization outcomes and limit functional recovery in people with schizophrenia. While deficits in social skills and social cognition contribute to much of the impairment in social functioning in schizophrenia, our focus on the current review is social motivation-the drive to connect with others and form meaningful, lasting relationships. We pay particular attention to how recent research on reward informs, and limits, our understanding of the construct. Recent findings that parse out key components of human motivation, especially the temporal nature of reward and effort, are informative for understanding some aspects of social motivation. This approach, however, fails to fully integrate the critical influence of uncertainty and punishment (e.g., avoidance, threat) in social motivation. In the current review, we argue for the importance of experimental paradigms and real-time measurement to capture the interaction between social approach and avoidance in characterizing social affiliation in schizophrenia. We end with suggestions for how researchers might move the field forward by emphasizing the ecological validity of social motivation paradigms, including dynamic, momentary assessment of social reward and punishment using mobile technology and other innovative tools. Copyright © 2018. Published by Elsevier Ltd.
-
Differences in reward processing between putative cell types in primate prefrontal cortex
Fan, Hongwei; Wang, Rubin; Sakagami, Masamichi
2017-01-01
Single-unit studies in monkeys have demonstrated that neurons in the prefrontal cortex predict the reward type, reward amount or reward availability associated with a stimulus. To examine contributions of pyramidal cells and interneurons in reward processing, single-unit activity was extracellularly recorded in prefrontal cortices of four monkeys performing a reward prediction task. Based on their shapes of spike waveforms, prefrontal neurons were classified into broad-spike and narrow-spike units that represented putative pyramidal cells and interneurons, respectively. We mainly observed that narrow-spike neurons showed higher firing rates but less bursty discharges than did broad-spike neurons. Both narrow-spike and broad-spike cells selectively responded to the stimulus, reward and their interaction, and the proportions of each type of selective neurons were similar between the two cell classes. Moreover, the two types of cells displayed equal reliability of reward or stimulus discrimination. Furthermore, we found that broad-spike and narrow-spike cells showed distinct mechanisms for encoding reward or stimulus information. Broad-spike neurons raised their firing rate relative to the baseline rate to represent the preferred reward or stimulus information, whereas narrow-spike neurons inhibited their firing rate lower than the baseline rate to encode the non-preferred reward or stimulus information. Our results suggest that narrow-spike and broad-spike cells were equally involved in reward and stimulus processing in the prefrontal cortex. They utilized a binary strategy to complementarily represent reward or stimulus information, which was consistent with the task structure in which the monkeys were required to remember two reward conditions and two visual stimuli. PMID:29261734
-
Differences in reward processing between putative cell types in primate prefrontal cortex.
Fan, Hongwei; Pan, Xiaochuan; Wang, Rubin; Sakagami, Masamichi
2017-01-01
Single-unit studies in monkeys have demonstrated that neurons in the prefrontal cortex predict the reward type, reward amount or reward availability associated with a stimulus. To examine contributions of pyramidal cells and interneurons in reward processing, single-unit activity was extracellularly recorded in prefrontal cortices of four monkeys performing a reward prediction task. Based on their shapes of spike waveforms, prefrontal neurons were classified into broad-spike and narrow-spike units that represented putative pyramidal cells and interneurons, respectively. We mainly observed that narrow-spike neurons showed higher firing rates but less bursty discharges than did broad-spike neurons. Both narrow-spike and broad-spike cells selectively responded to the stimulus, reward and their interaction, and the proportions of each type of selective neurons were similar between the two cell classes. Moreover, the two types of cells displayed equal reliability of reward or stimulus discrimination. Furthermore, we found that broad-spike and narrow-spike cells showed distinct mechanisms for encoding reward or stimulus information. Broad-spike neurons raised their firing rate relative to the baseline rate to represent the preferred reward or stimulus information, whereas narrow-spike neurons inhibited their firing rate lower than the baseline rate to encode the non-preferred reward or stimulus information. Our results suggest that narrow-spike and broad-spike cells were equally involved in reward and stimulus processing in the prefrontal cortex. They utilized a binary strategy to complementarily represent reward or stimulus information, which was consistent with the task structure in which the monkeys were required to remember two reward conditions and two visual stimuli.
-
Richards, Jessica M.; Plate, Rista C.; Ernst, Monique
2013-01-01
The neural systems underlying reward-related behaviors across development have recently generated a great amount of interest. Yet, the neurodevelopmental literature on reward processing is marked by inconsistencies due to the heterogeneity of the reward paradigms used, the complexity of the behaviors being studied, and the developing brain itself as a moving target. The present review will examine task design as one source of variability across findings by compiling this literature along three dimensions: (1) task structures, (2) cognitive processes, and (3) neural systems. We start with the presentation of a heuristic neural systems model, the Triadic Model, as a way to provide a theoretical framework for the neuroscience research on motivated behaviors. We then discuss the principles guiding reward task development. Finally, we review the extant developmental neuroimaging literature on reward-related processing, organized by reward task type. We hope that this approach will help to clarify the literature on the functional neurodevelopment of reward-related neural systems, and to identify the role of the experimental parameters that significantly influence these findings. PMID:23518270
-
Reward deficiency and anti-reward in pain chronification.
Borsook, D; Linnman, C; Faria, V; Strassman, A M; Becerra, L; Elman, I
2016-09-01
Converging lines of evidence suggest that the pathophysiology of pain is mediated to a substantial degree via allostatic neuroadaptations in reward- and stress-related brain circuits. Thus, reward deficiency (RD) represents a within-system neuroadaptation to pain-induced protracted activation of the reward circuits that leads to depletion-like hypodopaminergia, clinically manifested anhedonia, and diminished motivation for natural reinforcers. Anti-reward (AR) conversely pertains to a between-systems neuroadaptation involving over-recruitment of key limbic structures (e.g., the central and basolateral amygdala nuclei, the bed nucleus of the stria terminalis, the lateral tegmental noradrenergic nuclei of the brain stem, the hippocampus and the habenula) responsible for massive outpouring of stressogenic neurochemicals (e.g., norepinephrine, corticotropin releasing factor, vasopressin, hypocretin, and substance P) giving rise to such negative affective states as anxiety, fear and depression. We propose here the Combined Reward deficiency and Anti-reward Model (CReAM), in which biopsychosocial variables modulating brain reward, motivation and stress functions can interact in a 'downward spiral' fashion to exacerbate the intensity, chronicity and comorbidities of chronic pain syndromes (i.e., pain chronification). Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.
-
Ostrander, Serena; Cazares, Victor A.; Kim, Charissa; Cheung, Shauna; Gonzalez, Isabel; Izquierdo, Alicia
2011-01-01
The orbitofrontal cortex (OFC) and basolateral nucleus of the amygdala (BLA) are important neural regions in responding adaptively to changes in the incentive value of reward. Recent evidence suggests these structures may be differentially engaged in effort and cue-guided choice behavior. In two t-maze experiments, we examined the effects of bilateral lesions of either BLA or OFC on 1) effortful choices where rats could climb a barrier for a high reward or select a low reward with no effort and 2) effortful choices when a visual cue signaled changes in reward magnitude. In both experiments, BLA rats displayed transient work aversion, choosing the effortless low reward option. OFC rats were work averse only in the no cue conditions, displaying a pattern of attenuated recovery from the cue conditions signaling reward unavailability in the effortful arm. Control measures rule out an inability to discriminate the cue in either lesion group. PMID:21639604
-
Kenkel, W M; Yee, J R; Moore, K; Madularu, D; Kulkarni, P; Gamber, K; Nedelman, M; Ferris, C F
2016-03-22
Anxiety and social deficits, often involving communication impairment, are fundamental clinical features of fragile X syndrome. There is growing evidence that dysregulation in reward processing is a contributing factor to the social deficits observed in many psychiatric disorders. Hence, we hypothesized that transgenic fragile X mental retardation 1 gene (fmr1) KO (FX) rats would display alterations in reward processing. To this end, awake control and FX rats were imaged for changes in blood oxygen level dependent (BOLD) signal intensity in response to the odor of almond, a stimulus to elicit the innate reward response. Subjects were 'odor naive' to this evolutionarily conserved stimulus. The resulting changes in brain activity were registered to a three-dimensional segmented, annotated rat atlas delineating 171 brain regions. Both wild-type (WT) and FX rats showed robust brain activation to a rewarding almond odor, though FX rats showed an altered temporal pattern and tended to have a higher number of voxels with negative BOLD signal change from baseline. This pattern of greater negative BOLD was especially apparent in the Papez circuit, critical to emotional processing and the mesolimbic/habenular reward circuit. WT rats showed greater positive BOLD response in the supramammillary area, whereas FX rats showed greater positive BOLD response in the dorsal lateral striatum, and greater negative BOLD response in the retrosplenial cortices, the core of the accumbens and the lateral preoptic area. When tested in a freely behaving odor-investigation paradigm, FX rats failed to show the preference for almond odor which typifies WT rats. However, FX rats showed investigation profiles similar to WT when presented with social odors. These data speak to an altered processing of this highly salient novel odor in the FX phenotype and lend further support to the notion that altered reward systems in the brain may contribute to fragile X syndrome symptomology.
-
Encoding of marginal utility across time in the human brain
Pine, Alex; Seymour, Ben; Roiser, Jonathan P; Bossaerts, Peter; Friston, Karl J.; Curran, H. Valerie; Dolan, Raymond J.
2010-01-01
Marginal utility theory prescribes the relationship between the objective property of the magnitude of rewards and their subjective value. Despite its pervasive influence, however, there is remarkably little direct empirical evidence for such a theory of value, let alone of its neurobiological basis. We show that human preferences in an inter-temporal choice task are best described by a model that integrates marginally diminishing utility with temporal discounting. Using functional magnetic resonance imaging (fMRI), we show that activity in the dorsal striatum encodes both the marginal utility of rewards, over and above that which can be described by their magnitude alone, and the discounting associated with increasing time. In addition, our data show that dorsal striatum may be involved in integrating subjective valuation systems inherent to time and magnitude, thereby providing an overall metric of value used to guide choice behaviour. Furthermore, during choice we show that anterior cingulate activity correlates with the degree of difficulty associated with dissonance between value and time. Our data support an integrative architecture for decision-making, revealing the neural representation of distinct subcomponents of value that may contribute to impulsivity and decisiveness. PMID:19641120
-
Szabo, Miruna; Deco, Gustavo; Fusi, Stefano; Del Giudice, Paolo; Mattia, Maurizio; Stetter, Martin
2006-05-01
Recent experiments on behaving monkeys have shown that learning a visual categorization task makes the neurons in infero-temporal cortex (ITC) more selective to the task-relevant features of the stimuli (Sigala and Logothetis in Nature 415 318-320, 2002). We hypothesize that such a selectivity modulation emerges from the interaction between ITC and other cortical area, presumably the prefrontal cortex (PFC), where the previously learned stimulus categories are encoded. We propose a biologically inspired model of excitatory and inhibitory spiking neurons with plastic synapses, modified according to a reward based Hebbian learning rule, to explain the experimental results and test the validity of our hypothesis. We assume that the ITC neurons, receiving feature selective inputs, form stronger connections with the category specific neurons to which they are consistently associated in rewarded trials. After learning, the top-down influence of PFC neurons enhances the selectivity of the ITC neurons encoding the behaviorally relevant features of the stimuli, as observed in the experiments. We conclude that the perceptual representation in visual areas like ITC can be strongly affected by the interaction with other areas which are devoted to higher cognitive functions.
-
Reward for food odors: an fMRI study of liking and wanting as a function of metabolic state and BMI.
Jiang, Tao; Soussignan, Robert; Schaal, Benoist; Royet, Jean-Pierre
2015-04-01
Brain reward systems mediate liking and wanting for food reward. Here, we explore the differential involvement of the following structures for these two components: the ventral and dorsal striatopallidal area, orbitofrontal cortex (OFC), anterior insula and anterior cingulate. Twelve healthy female participants were asked to rate pleasantness (liking of food and non-food odors) and the desire to eat (wanting of odor-evoked food) during event-related functional magnetic resonance imaging (fMRI). The subjective ratings and fMRI were performed in hunger and satiety states. Activations of regions of interest were compared as a function of task (liking vs wanting), odor category (food vs non-food) and metabolic state (hunger vs satiety). We found that the nucleus accumbens and ventral pallidum were differentially involved in liking or wanting during the hunger state, which suggests a reciprocal inhibitory influence between these structures. Neural activation of OFC subregions was correlated with either liking or wanting ratings, suggesting an OFC role in reward processing magnitude. Finally, during the hunger state, participants with a high body mass index exhibited less activation in neural structures underlying food reward processing. Our results suggest that food liking and wanting are two separable psychological constructs and may be functionally segregated within the cortico-striatopallidal circuit. © The Author (2014). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
Ventral pallidal encoding of reward-seeking behavior depends on the underlying associative structure
Stout, Nakura; Acs, Deanna
2018-01-01
Despite its being historically conceptualized as a motor expression site, emerging evidence suggests the ventral pallidum (VP) plays a more active role in integrating information to generate motivation. Here, we investigated whether rat VP cue responses would encode and contribute similarly to the vigor of reward-seeking behaviors trained under Pavlovian versus instrumental contingencies, when these behavioral responses consist of superficially similar locomotor response patterns but may reflect distinct underlying decision-making processes. We find that cue-elicited activity in many VP neurons predicts the latency of instrumental reward seeking, but not of Pavlovian response latency. Further, disruption of VP signaling increases the latency of instrumental but not Pavlovian reward seeking. This suggests that VP encoding of and contributions to response vigor are specific to the ability of incentive cues to invigorate reward-seeking behaviors upon which reward delivery is contingent. PMID:29565248
-
Bhalla, Upinder S.; Hellgren Kotaleski, Jeanette
2016-01-01
In reward learning, the integration of NMDA-dependent calcium and dopamine by striatal projection neurons leads to potentiation of corticostriatal synapses through CaMKII/PP1 signaling. In order to elicit the CaMKII/PP1-dependent response, the calcium and dopamine inputs should arrive in temporal proximity and must follow a specific (dopamine after calcium) order. However, little is known about the cellular mechanism which enforces these temporal constraints on the signal integration. In this computational study, we propose that these temporal requirements emerge as a result of the coordinated signaling via two striatal phosphoproteins, DARPP-32 and ARPP-21. Specifically, DARPP-32-mediated signaling could implement an input-interval dependent gating function, via transient PP1 inhibition, thus enforcing the requirement for temporal proximity. Furthermore, ARPP-21 signaling could impose the additional input-order requirement of calcium and dopamine, due to its Ca2+/calmodulin sequestering property when dopamine arrives first. This highlights the possible role of phosphoproteins in the temporal aspects of striatal signal transduction. PMID:27584878
-
Cohen, Michael S.; Rissman, Jesse; Suthana, Nanthia A.; Castel, Alan D.; Knowlton, Barbara J.
2014-01-01
A number of prior fMRI studies have focused on the ways in which the midbrain dopaminergic reward system co-activates with hippocampus to potentiate memory for valuable items. However, another means by which people could selectively remember more valuable to-be-remembered items is to be selective in their use of effective but effortful encoding strategies. To broadly examine the neural mechanisms of value on subsequent memory, we used fMRI to examine how differences in brain activity at encoding as a function of value relate to subsequent free recall for words. Each word was preceded by an arbitrarily assigned point value, and participants went through multiple study-test cycles with feedback on their point total at the end of each list, allowing for sculpting of cognitive strategies. We examined the correlation between value-related modulation of brain activity and participants’ selectivity index, a measure of how close participants were to their optimal point total given the number of items recalled. Greater selectivity scores were associated with greater differences in activation of semantic processing regions, including left inferior frontal gyrus and left posterior lateral temporal cortex, during encoding of high-value words relative to low-value words. Although we also observed value-related modulation within midbrain and ventral striatal reward regions, our fronto-temporal findings suggest that strategic engagement of deep semantic processing may be an important mechanism for selectively encoding valuable items. PMID:24683066
-
Story, Giles W.; Vlaev, Ivo; Seymour, Ben; Darzi, Ara; Dolan, Raymond J.
2014-01-01
The tendency to make unhealthy choices is hypothesized to be related to an individual's temporal discount rate, the theoretical rate at which they devalue delayed rewards. Furthermore, a particular form of temporal discounting, hyperbolic discounting, has been proposed to explain why unhealthy behavior can occur despite healthy intentions. We examine these two hypotheses in turn. We first systematically review studies which investigate whether discount rates can predict unhealthy behavior. These studies reveal that high discount rates for money (and in some instances food or drug rewards) are associated with several unhealthy behaviors and markers of health status, establishing discounting as a promising predictive measure. We secondly examine whether intention-incongruent unhealthy actions are consistent with hyperbolic discounting. We conclude that intention-incongruent actions are often triggered by environmental cues or changes in motivational state, whose effects are not parameterized by hyperbolic discounting. We propose a framework for understanding these state-based effects in terms of the interplay of two distinct reinforcement learning mechanisms: a “model-based” (or goal-directed) system and a “model-free” (or habitual) system. Under this framework, while discounting of delayed health may contribute to the initiation of unhealthy behavior, with repetition, many unhealthy behaviors become habitual; if health goals then change, habitual behavior can still arise in response to environmental cues. We propose that the burgeoning development of computational models of these processes will permit further identification of health decision-making phenotypes. PMID:24659960
-
Story, Giles W; Vlaev, Ivo; Seymour, Ben; Darzi, Ara; Dolan, Raymond J
2014-01-01
The tendency to make unhealthy choices is hypothesized to be related to an individual's temporal discount rate, the theoretical rate at which they devalue delayed rewards. Furthermore, a particular form of temporal discounting, hyperbolic discounting, has been proposed to explain why unhealthy behavior can occur despite healthy intentions. We examine these two hypotheses in turn. We first systematically review studies which investigate whether discount rates can predict unhealthy behavior. These studies reveal that high discount rates for money (and in some instances food or drug rewards) are associated with several unhealthy behaviors and markers of health status, establishing discounting as a promising predictive measure. We secondly examine whether intention-incongruent unhealthy actions are consistent with hyperbolic discounting. We conclude that intention-incongruent actions are often triggered by environmental cues or changes in motivational state, whose effects are not parameterized by hyperbolic discounting. We propose a framework for understanding these state-based effects in terms of the interplay of two distinct reinforcement learning mechanisms: a "model-based" (or goal-directed) system and a "model-free" (or habitual) system. Under this framework, while discounting of delayed health may contribute to the initiation of unhealthy behavior, with repetition, many unhealthy behaviors become habitual; if health goals then change, habitual behavior can still arise in response to environmental cues. We propose that the burgeoning development of computational models of these processes will permit further identification of health decision-making phenotypes.
-
Cohen, Michael S; Rissman, Jesse; Suthana, Nanthia A; Castel, Alan D; Knowlton, Barbara J
2014-06-01
A number of prior fMRI studies have focused on the ways in which the midbrain dopaminergic reward system coactivates with hippocampus to potentiate memory for valuable items. However, another means by which people could selectively remember more valuable to-be-remembered items is to be selective in their use of effective but effortful encoding strategies. To broadly examine the neural mechanisms of value on subsequent memory, we used fMRI to assess how differences in brain activity at encoding as a function of value relate to subsequent free recall for words. Each word was preceded by an arbitrarily assigned point value, and participants went through multiple study-test cycles with feedback on their point total at the end of each list, allowing for sculpting of cognitive strategies. We examined the correlation between value-related modulation of brain activity and participants' selectivity index, which measures how close participants were to their optimal point total, given the number of items recalled. Greater selectivity scores were associated with greater differences in the activation of semantic processing regions, including left inferior frontal gyrus and left posterior lateral temporal cortex, during the encoding of high-value words relative to low-value words. Although we also observed value-related modulation within midbrain and ventral striatal reward regions, our fronto-temporal findings suggest that strategic engagement of deep semantic processing may be an important mechanism for selectively encoding valuable items.
-
Brain intrinsic network connectivity in individuals with frequent tanning behavior.
Ketcherside, Ariel; Filbey, Francesca M; Aubert, Pamela M; Seibyl, John P; Price, Julianne L; Adinoff, Bryon
2018-05-01
Emergent studies suggest a bidirectional relationship between brain functioning and the skin. This neurocutaneous connection may be responsible for the reward response to tanning and, thus, may contribute to excessive tanning behavior. To date, however, this association has not yet been examined. To explore whether intrinsic brain functional connectivity within the default mode network (DMN) is related to indoor tanning behavior. Resting state functional connectivity (rsFC) was obtained in twenty adults (16 females) with a history of indoor tanning. Using a seed-based [(posterior cingulate cortex (PCC)] approach, the relationship between tanning severity and FC strength was assessed. Tanning severity was measured with symptom count from the Structured Clinical Interview for Tanning Abuse and Dependence (SITAD) and tanning intensity (lifetime indoor tanning episodes/years tanning). rsFC strength between the PCC and other DMN regions (left globus pallidus, left medial frontal gyrus, left superior frontal gyrus) is positively correlated with tanning symptom count. rsFC strength between the PCC and salience network regions (right anterior cingulate cortex, left inferior parietal lobe, left inferior temporal gyrus) is correlated with tanning intensity. Greater connectivity between tanning severity and DMN and salience network connectivity suggests that heightened self-awareness of salient stimuli may be a mechanism that underlies frequent tanning behavior. These findings add to the growing evidence of brain-skin connection and reflect dysregulation in the reward processing networks in those with frequent tanning.
-
Bloch, Guy; Bar-Shai, Noam; Cytter, Yotam; Green, Rachel
2017-11-19
The interactions between flowering plants and insect pollinators shape ecological communities and provide one of the best examples of coevolution. Although these interactions have received much attention in both ecology and evolution, their temporal aspects are little explored. Here we review studies on the circadian organization of pollination-related traits in bees and flowers. Research, mostly with the honeybee, Apis mellifera , has implicated the circadian clock in key aspects of their foraging for flower rewards. These include anticipation, timing of visits to flowers at specified locations and time-compensated sun-compass orientation. Floral rhythms in traits such as petal opening, scent release and reward availability also show robust daily rhythms. However, in only few studies was it possible to adequately determine whether these oscillations are driven by external time givers such as light and temperature cycles, or endogenous circadian clocks. The interplay between the timing of flower and pollinator rhythms may be ecologically significant. Circadian regulation of pollination-related traits in only few species may influence the entire pollination network and thus affect community structure and local biodiversity. We speculate that these intricate chronobiological interactions may be vulnerable to anthropogenic effects such as the introduction of alien invasive species, pesticides or environmental pollutants.This article is part of the themed issue 'Wild clocks: integrating chronobiology and ecology to understand timekeeping in free-living animals'. © 2017 The Author(s).
-
Hippocampal neural correlates for values of experienced events.
Lee, Hyunjung; Ghim, Jeong-Wook; Kim, Hoseok; Lee, Daeyeol; Jung, MinWhan
2012-10-24
Newly experienced events are often remembered together with how rewarding the experiences are personally. Although the hippocampus is a candidate structure where subjective values are integrated with other elements of episodic memory, it is uncertain whether and how the hippocampus processes value-related information. We examined how activity of dorsal CA1 and dorsal subicular neurons in rats performing a dynamic foraging task was related to reward values that were estimated using a reinforcement learning model. CA1 neurons carried significant signals related to action values before the animal revealed its choice behaviorally, indicating that the information on the expected values of potential choice outcomes was available in CA1. Moreover, after the outcome of the animal's goal choice was revealed, CA1 neurons carried robust signals for the value of chosen action and they temporally overlapped with the signals related to the animal's goal choice and its outcome, indicating that all the signals necessary to evaluate the outcome of an experienced event converged in CA1. On the other hand, value-related signals were substantially weaker in the subiculum. These results suggest a major role of CA1 in adding values to experienced events during episodic memory encoding. Given that CA1 neuronal activity is modulated by diverse attributes of an experienced event, CA1 might be a place where all the elements of episodic memory are integrated.
-
Does Goal Relevant Episodic Future Thinking Amplify the Effect on Delay Discounting?
O’Donnell, Sara; Daniel, Tinuke Oluyomi; Epstein, Leonard H.
2017-01-01
Delay discounting (DD) is the preference for smaller immediate rewards over larger delayed rewards. Research shows episodic future thinking (EFT), or mentally simulating future experiences, reframes the choice between small immediate and larger delayed rewards, and can reduce DD. Only general EFT has been studied, whereby people reframe decisions in terms of non-goal related future events. Since future thinking is often goal-oriented and leads to greater activation of brain regions involved in prospection, goal-oriented EFT may be associated with greater reductions in DD than general goal-unrelated EFT. The present study (n = 104, Mage = 22.25, SD = 3.42; 50% Female) used a between-subjects 2×2 factorial design with type of episodic thinking (Goal, General) and temporal perspective (Episodic future versus recent thinking; EFT vs ERT) as between factors. Results showed a significant reduction in DD for EFT groups (p < 0.001, Cohen’s d effect size = 0.89), and goal-EFT was more effective than general-EFT on reducing DD (p = 0.03, d = 0.64). PMID:28282631
-
The neuroscience of positive memory deficits in depression
Dillon, Daniel G.
2015-01-01
Adults with unipolar depression typically show poor episodic memory for positive material, but the neuroscientific mechanisms responsible for this deficit have not been characterized. I suggest a simple hypothesis: weak memory for positive material in depression reflects disrupted communication between the mesolimbic dopamine pathway and medial temporal lobe (MTL) memory systems during encoding. This proposal draws on basic research showing that dopamine release in the hippocampus is critical for the transition from early- to late-phase long-term potentiation (LTP) that marks the conversion of labile, short-term memories into stable, long-term memories. Neuroimaging and pharmacological data from healthy humans paint a similar picture: activation of the mesolimbic reward circuit enhances encoding and boosts retention. Unipolar depression is characterized by anhedonia–loss of pleasure–and reward circuit dysfunction, which is believed to reflect negative effects of stress on the mesolimbic dopamine pathway. Thus, I propose that the MTL is deprived of strengthening reward signals in depressed adults and memory for positive events suffers accordingly. Although other mechanisms are important, this hypothesis holds promise as an explanation for positive memory deficits in depression. PMID:26441703
-
Does goal relevant episodic future thinking amplify the effect on delay discounting?
O'Donnell, Sara; Oluyomi Daniel, Tinuke; Epstein, Leonard H
2017-05-01
Delay discounting (DD) is the preference for smaller immediate rewards over larger delayed rewards. Research shows episodic future thinking (EFT), or mentally simulating future experiences, reframes the choice between small immediate and larger delayed rewards, and can reduce DD. Only general EFT has been studied, whereby people reframe decisions in terms of non-goal related future events. Since future thinking is often goal-oriented and leads to greater activation of brain regions involved in prospection, goal-oriented EFT may be associated with greater reductions in DD than general goal-unrelated EFT. The present study (n=104, M age =22.25, SD=3.42; 50% Female) used a between-subjects 2×2 factorial design with type of episodic thinking (Goal, General) and temporal perspective (Episodic future versus recent thinking; EFT vs ERT) as between factors. Results showed a significant reduction in DD for EFT groups (p<0.001, Cohen's d effect size=0.89), and goal-EFT was more effective than general-EFT on reducing DD (p=0.03, d=0.64). Copyright © 2017 Elsevier Inc. All rights reserved.
-
Reward Structures in the Public High School
ERIC Educational Resources Information Center
Spuck, Dennis W.
1974-01-01
Examines the relationship between eight categories of rewards available to teachers in high schools and teacher behaviors of absenteeism, recruitment, and retention. Findings emphasize the importance of intrinsic motivators in professional organizations and suggest basic differences in motivational patterns between professional and production…
-
NASA Technical Reports Server (NTRS)
HolmesParker, Chris; Taylor, Mathew E.; Tumer, Kagan; Agogino, Adrian
2014-01-01
Learning in multiagent systems can be slow because agents must learn both how to behave in a complex environment and how to account for the actions of other agents. The inability of an agent to distinguish between the true environmental dynamics and those caused by the stochastic exploratory actions of other agents creates noise in each agent's reward signal. This learning noise can have unforeseen and often undesirable effects on the resultant system performance. We define such noise as exploratory action noise, demonstrate the critical impact it can have on the learning process in multiagent settings, and introduce a reward structure to effectively remove such noise from each agent's reward signal. In particular, we introduce Coordinated Learning without Exploratory Action Noise (CLEAN) rewards and empirically demonstrate their benefits
-
Reward Circuitry in Addiction.
Cooper, Sarah; Robison, A J; Mazei-Robison, Michelle S
2017-07-01
Understanding the brain circuitry that underlies reward is critical to improve treatment for many common health issues, including obesity, depression, and addiction. Here we focus on insights into the organization and function of reward circuitry and its synaptic and structural adaptations in response to cocaine exposure. While the importance of certain circuits, such as the mesocorticolimbic dopamine pathway, are well established in drug reward, recent studies using genetics-based tools have revealed functional changes throughout the reward circuitry that contribute to different facets of addiction, such as relapse and craving. The ability to observe and manipulate neuronal activity within specific cell types and circuits has led to new insight into not only the basic connections between brain regions, but also the molecular changes within these specific microcircuits, such as neurotrophic factor and GTPase signaling or α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptor function, that underlie synaptic and structural plasticity evoked by drugs of abuse. Excitingly, these insights from preclinical rodent work are now being translated into the clinic, where transcranial magnetic simulation and deep brain stimulation therapies are being piloted in human cocaine dependence. Thus, this review seeks to summarize current understanding of the major brain regions implicated in drug-related behaviors and the molecular mechanisms that contribute to altered connectivity between these regions, with the postulation that increased knowledge of the plasticity within the drug reward circuit will lead to new and improved treatments for addiction.
-
A Simple Network Architecture Accounts for Diverse Reward Time Responses in Primary Visual Cortex.
Huertas, Marco A; Hussain Shuler, Marshall G; Shouval, Harel Z
2015-09-16
Many actions performed by animals and humans depend on an ability to learn, estimate, and produce temporal intervals of behavioral relevance. Exemplifying such learning of cued expectancies is the observation of reward-timing activity in the primary visual cortex (V1) of rodents, wherein neural responses to visual cues come to predict the time of future reward as behaviorally experienced in the past. These reward-timing responses exhibit significant heterogeneity in at least three qualitatively distinct classes: sustained increase or sustained decrease in firing rate until the time of expected reward, and a class of cells that reach a peak in firing at the expected delay. We elaborate upon our existing model by including inhibitory and excitatory units while imposing simple connectivity rules to demonstrate what role these inhibitory elements and the simple architectures play in sculpting the response dynamics of the network. We find that simply adding inhibition is not sufficient for obtaining the different distinct response classes, and that a broad distribution of inhibitory projections is necessary for obtaining peak-type responses. Furthermore, although changes in connection strength that modulate the effects of inhibition onto excitatory units have a strong impact on the firing rate profile of these peaked responses, the network exhibits robustness in its overall ability to predict the expected time of reward. Finally, we demonstrate how the magnitude of expected reward can be encoded at the expected delay in the network and how peaked responses express this reward expectancy. Heterogeneity in single-neuron responses is a common feature of neuronal systems, although sometimes, in theoretical approaches, it is treated as a nuisance and seldom considered as conveying a different aspect of a signal. In this study, we focus on the heterogeneous responses in the primary visual cortex of rodents trained with a predictable delayed reward time. We describe under what conditions this heterogeneity can arise by self-organization, and what information it can convey. This study, while focusing on a specific system, provides insight onto how heterogeneity can arise in general while also shedding light onto mechanisms of reinforcement learning using realistic biological assumptions. Copyright © 2015 the authors 0270-6474/15/3512659-14$15.00/0.
-
Within-subject neural reactivity to reward and threat is inverted in young adolescents.
Thomason, M E; Marusak, H A
2017-07-01
As children mature, they become increasingly independent and less reliant on caregiver support. Changes in brain systems are likely to stimulate and guide this process. One mechanistic hypothesis suggests that changes in neural systems that process reward and threat support the increase in exploratory behavior observed in the transition to adolescence. This study examines the basic tenets of this hypothesis by performing functional magnetic resonance imaging (fMRI) during well-established reward and threat processing tasks in 40 children and adolescents, aged 9-15 years. fMRI responses in the striatum and amygdala are fit to a model predicting that striatal reward and amygdala threat-responses will be unrelated in younger participants (aged 9-12 years), while older participants (aged 13-15 years) will differentially engage these structures. Our data are consistent with this model. Activity in the striatum and amygdala are comparable in younger children, but in older children, they are inversely related; those more responsive to reward show a reduced threat-response. Analyses testing age as a continuous variable yield consistent results. In addition, the proportion of threat to reward-response relates to self-reported approach behavior in older but not younger youth, exposing behavioral relevance in the relative level of activity in these structures. Results are consistent with the notion that both individual and developmental differences drive reward-seeking behavior in adolescence. While these response patterns may serve adaptive functions in the shift to independence, skew in these systems may relate to increased rates of emotional psychopathology and risk-taking observed in adolescence.
-
Amygdala mu-opioid receptors mediate the motivating influence of cue-triggered reward expectations.
Lichtenberg, Nina T; Wassum, Kate M
2017-02-01
Environmental reward-predictive stimuli can retrieve from memory a specific reward expectation that allows them to motivate action and guide choice. This process requires the basolateral amygdala (BLA), but little is known about the signaling systems necessary within this structure. Here we examined the role of the neuromodulatory opioid receptor system in the BLA in such cue-directed action using the outcome-specific Pavlovian-to-instrumental transfer (PIT) test in rats. Inactivation of BLA mu-, but not delta-opioid receptors was found to dose-dependently attenuate the ability of a reward-predictive cue to selectively invigorate the performance of actions directed at the same unique predicted reward (i.e. to express outcome-specific PIT). BLA mu-opioid receptor inactivation did not affect the ability of a reward itself to similarly motivate action (outcome-specific reinstatement), suggesting a more selective role for the BLA mu-opioid receptor in the motivating influence of currently unobservable rewarding events. These data reveal a new role for BLA mu-opioid receptor activation in the cued recall of precise reward memories and the use of this information to motivate specific action plans. © 2016 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.
-
Richards, Jessica M; Plate, Rista C; Ernst, Monique
2013-06-01
The neural systems underlying reward-related behaviors across development have recently generated a great amount of interest. Yet, the neurodevelopmental literature on reward processing is marked by inconsistencies due to the heterogeneity of the reward paradigms used, the complexity of the behaviors being studied, and the developing brain itself as a moving target. The present review will examine task design as one source of variability across findings by compiling this literature along three dimensions: (1) task structures, (2) cognitive processes, and (3) neural systems. We start with the presentation of a heuristic neural systems model, the Triadic Model, as a way to provide a theoretical framework for the neuroscience research on motivated behaviors. We then discuss the principles guiding reward task development. Finally, we review the extant developmental neuroimaging literature on reward-related processing, organized by reward task type. We hope that this approach will help to clarify the literature on the functional neurodevelopment of reward-related neural systems, and to identify the role of the experimental parameters that significantly influence these findings. Published by Elsevier Ltd.
-
Temporal patterns of apparent leg band retention in North American geese
Zimmerman, Guthrie S.; Kendall, William L.; Moser, Timothy J.; White, Gary C.; Doherty, Paul F.
2009-01-01
An important assumption of mark?recapture studies is that individuals retain their marks, which has not been assessed for goose reward bands. We estimated aluminum leg band retention probabilities and modeled how band retention varied with band type (standard vs. reward band), band age (1-40 months), and goose characteristics (species and size class) for Canada (Branta canadensis), cackling (Branta hutchinsii), snow (Chen caerulescens), and Ross?s (Chen rossii) geese that field coordinators double-leg banded during a North American goose reward band study (N = 40,999 individuals from 15 populations). We conditioned all models in this analysis on geese that were encountered with >1 leg band still attached (n = 5,747 dead recoveries and live recaptures). Retention probabilities for standard aluminum leg bands were high (estimate of 0.9995, SE = 0.001) and constant over 1-40 months. In contrast, apparent retention probabilities for reward bands demonstrated an interactive relationship between 5 size and species classes (small cackling, medium Canada, large Canada, snow, and Ross?s geese). In addition, apparent retention probabilities for each of the 5 classes varied quadratically with time, being lower immediately after banding and at older age classes. The differential retention probabilities among band type (reward vs. standard) that we observed suggests that 1) models estimating reporting probability should incorporate differential band loss if it is nontrivial, 2) goose managers should consider the costs and benefits of double-banding geese on an operational basis, and 3) the United States Geological Survey Bird Banding Lab should modify protocols for receiving recovery data.
-
Nurse practitioners: leadership behaviors and organizational climate.
Jones, L C; Guberski, T D; Soeken, K L
1990-01-01
The purpose of this article is to examine the relationships of individual nurse practitioners' perceptions of the leadership climate in their organizations and self-reported formal and informal leadership behaviors. The nine climate dimensions (Structure, Responsibility, Reward, Perceived Support of Risk Taking, Warmth, Support, Standard Setting, Conflict, and Identity) identified by Litwin and Stringer in 1968 were used to predict five leadership dimensions (Meeting Organizational Needs, Managing Resources, Leadership Competence, Task Accomplishment, and Communications). Demographic variables of age, educational level, and percent of time spent performing administrative functions were forced as a first step in each multiple regression analysis and used to explain a significant amount of variance in all but one analysis. All leadership dimensions were predicted by at least one organizational climate dimension: (1) Meeting Organizational Needs by Risk and Reward; (2) Managing Resources by Risk and Structure; (3) Leadership Competence by Risk and Standards; (4) Task Accomplishment by Structure, Risk, and Standards; and (5) Communication by Rewards.
-
The Trouble with Pay for Performance.
ERIC Educational Resources Information Center
Young, I. Phillip
2003-01-01
Structured interviews with 575 administrators and supervisors in 6 school districts finds 8 impediments to school board implementation of an effective merit pay system for these employees: Lack of knowledge, teacher heritage, supervisor's ability, supervisor's motivation, managerial prerogatives, amount of rewards, and type of rewards. Offers…
-
Cornelius, Jack R; Aizenstein, Howard J; Chung, Tammy A; Douaihy, Antoine; Hayes, Jeanine; Daley, Dennis; Salloum, Ihsan M
Reward behavior, including reward behavior involving drugs, has been shown to be mediated by the ventral striatum and related structures of the reward system. The aim of this study was to assess reward-related activity as shown by fMRI before and after treatment among youth with comorbid cannabis dependence and major depression. We hypothesized that the reward task (Delgado et al., 2003) would elicit activation in the reward system, and that the level of activation in response to reward would increase from the beginning to the end of the 12-week treatment study as levels of depressive symptoms and cannabis use decreased. Six subjects were recruited from a larger treatment study in which all received Cognitive Behavioral Therapy/Motivational Enhancement Therapy (CBT/MET), and also were randomized to receive either fluoxetine or placebo. Each of the six subjects completed an fMRI card- guessing/reward task both before and after the 12-week treatment study. As hypothesized, the expected activation was noted for the reward task in the insula, prefrontal, and striatal areas, both before and after treatment. However, the participants showed lower reward-related activation after treatment relative to pre-treatment, which is opposite of what would be expected in depressed subjects who did not demonstrate a comorbid substance use disorder. These paradoxical findings suggest that the expected increase in activity for reward associated with treatment for depression was overshadowed by a decrease in reward-related activation associated with treatment of pathological cannabis use in these comorbid youth. These findings emphasize the importance of comorbid disorders in fMRI studies.
-
Affective traits and history of depression are related to ventral striatum connectivity.
DelDonno, Sophie R; Jenkins, Lisanne M; Crane, Natania A; Nusslock, Robin; Ryan, Kelly A; Shankman, Stewart A; Phan, K Luan; Langenecker, Scott A
2017-10-15
Studying remitted Major Depressive Disorder (rMDD) facilitates a better understanding of neural mechanisms for risk, given that confounding effects of active symptoms are removed. Disrupted functional connectivity has been reported in multiple networks in MDD. However, no study to date of rMDD has specifically examined connectivity of the ventral striatum (VS), a region highly implicated in reward and motivation. We investigated functional connectivity of the VS in individuals with and without a history of MDD, and in relation to affective personality traits. Forty-two individuals with rMDD and 28 healthy controls across two sites completed resting-state fMRI and the Behavioral Inhibition System/Behavioral Activation System Scale. Voxel-wise, whole-brain comparisons were conducted across and between groups for four seeds: left and right inferior VS (VSi), left and right superior VS (VSs). VSs connectivity to temporal and subcortical regions including the putamen and amygdala was positive and greater in HCs compared to rMDD individuals. Across groups, VSi connectivity was positively correlated with trait reward-responsiveness in somatomotor regions. Across groups, VSs connectivity was positively correlated with trait drive, particularly in the putamen, parahippocampal, and inferior temporal gyrus, and was negatively associated with trait behavioral inhibition in the anterior cingulate, frontal gyri, and insula. Limitations include scanning at two sites and using multiple comparisons. Group connectivity differences emerged from the VSs rather than VSi. VSs showed associations with trait drive and behavioral inhibition, whereas VSi corrrelated with reward-responsiveness. Depression history and affective traits contribute meaningful and specific information about VS connectivity in understanding risk for MDD. Copyright © 2017 Elsevier B.V. All rights reserved.
-
Regional brain responses in nulliparous women to emotional infant stimuli.
Montoya, Jessica L; Landi, Nicole; Kober, Hedy; Worhunsky, Patrick D; Rutherford, Helena J V; Mencl, W Einar; Mayes, Linda C; Potenza, Marc N
2012-01-01
Infant cries and facial expressions influence social interactions and elicit caretaking behaviors from adults. Recent neuroimaging studies suggest that neural responses to infant stimuli involve brain regions that process rewards. However, these studies have yet to investigate individual differences in tendencies to engage or withdraw from motivationally relevant stimuli. To investigate this, we used event-related fMRI to scan 17 nulliparous women. Participants were presented with novel infant cries of two distress levels (low and high) and unknown infant faces of varying affect (happy, sad, and neutral) in a randomized, counter-balanced order. Brain activation was subsequently correlated with scores on the Behavioral Inhibition System/Behavioral Activation System scale. Infant cries activated bilateral superior and middle temporal gyri (STG and MTG) and precentral and postcentral gyri. Activation was greater in bilateral temporal cortices for low- relative to high-distress cries. Happy relative to neutral faces activated the ventral striatum, caudate, ventromedial prefrontal, and orbitofrontal cortices. Sad versus neutral faces activated the precuneus, cuneus, and posterior cingulate cortex, and behavioral activation drive correlated with occipital cortical activations in this contrast. Behavioral inhibition correlated with activation in the right STG for high- and low-distress cries relative to pink noise. Behavioral drive correlated inversely with putamen, caudate, and thalamic activations for the comparison of high-distress cries to pink noise. Reward-responsiveness correlated with activation in the left precentral gyrus during the perception of low-distress cries relative to pink noise. Our findings indicate that infant cry stimuli elicit activations in areas implicated in auditory processing and social cognition. Happy infant faces may be encoded as rewarding, whereas sad faces activate regions associated with empathic processing. Differences in motivational tendencies may modulate neural responses to infant cues.
-
Sitek, Kevin R; Cai, Shanqing; Beal, Deryk S; Perkell, Joseph S; Guenther, Frank H; Ghosh, Satrajit S
2016-01-01
Persistent developmental stuttering is characterized by speech production disfluency and affects 1% of adults. The degree of impairment varies widely across individuals and the neural mechanisms underlying the disorder and this variability remain poorly understood. Here we elucidate compensatory mechanisms related to this variability in impairment using whole-brain functional and white matter connectivity analyses in persistent developmental stuttering. We found that people who stutter had stronger functional connectivity between cerebellum and thalamus than people with fluent speech, while stutterers with the least severe symptoms had greater functional connectivity between left cerebellum and left orbitofrontal cortex (OFC). Additionally, people who stutter had decreased functional and white matter connectivity among the perisylvian auditory, motor, and speech planning regions compared to typical speakers, but greater functional connectivity between the right basal ganglia and bilateral temporal auditory regions. Structurally, disfluency ratings were negatively correlated with white matter connections to left perisylvian regions and to the brain stem. Overall, we found increased connectivity among subcortical and reward network structures in people who stutter compared to controls. These connections were negatively correlated with stuttering severity, suggesting the involvement of cerebellum and OFC may underlie successful compensatory mechanisms by more fluent stutterers.
-
Structural connectivity of neural reward networks in youth at risk for substance use disorders.
Squeglia, Lindsay M; Sorg, Scott F; Jacobus, Joanna; Brumback, Ty; Taylor, Charles T; Tapert, Susan F
2015-07-01
Having a positive family history of alcohol use disorders (FHP), as well as aberrant reward circuitry, has been implicated in the initiation of substance use during adolescence. This study explored the relationship between FHP status and reward circuitry in substance naïve youth to better understand future risky behaviors. Participants were 49 FHP and 45 demographically matched family history negative (FHN) substance-naïve 12-14 year-olds (54 % female). Subjects underwent structural magnetic resonance imaging, including diffusion tensor imaging. Nucleus accumbens and orbitofrontal cortex volumes were derived using FreeSurfer, and FSL probabilistic tractography probed structural connectivity and differences in white matter diffusivity estimates (e.g. fractional anisotropy, and mean, radial, and axial diffusivity) between fiber tracts connecting these regions. FHP and FHN youth did not differ on nucleus accumbens or orbitofrontal cortex volumes, white matter tract volumes, or percentages of streamlines (a proxy for fiber tract count) connecting these regions. However, within white matter tracts connecting the nucleus accumbens to the orbitofrontal cortex, FHP youth had significantly lower mean and radial diffusivity (ps < 0.03) than FHN youth. While white matter macrostructure between salience and reward regions did not differ between FHP and FHN youth, FHP youth showed greater white matter coherence within these tracts than FHN youth. Aberrant connectivity between reward regions in FHP youth could be linked to an increased risk for substance use initiation.
-
Wang, Ziliang; Wu, Lingdan; Yuan, Kai; Hu, Yanbo; Zheng, Hui; Du, Xiaoxia; Dong, Guangheng
2018-06-08
Although online gaming may lead to Internet gaming disorder (IGD), most players are recreational game users (RGUs) who do not develop IGD. Thus far, little is known about brain structural abnormalities in IGD subjects relative to RGUs. The inclusion of RGUs as a control group could minimize the potential effects of gaming experience and gaming-related cue familiarity on the neural mechanism of IGD subjects. In the current study, structural magnetic resonance imaging data were acquired from 38 IGD subjects and 66 RGUs with comparable age, gender, and educational level. Group differences in cortical thickness and volume were analyzed using the FreeSurfer software. Correlations between cortical changes and addiction severity were calculated for both groups. Compared with the RGU group, the IGD group showed significantly decreased cortical thickness in the left lateral orbitofrontal cortex, inferior parietal lobule, bilateral cuneus, precentral gyrus, and right middle temporal gyrus. Moreover, significantly reduced cortical volume was observed in the left superior temporal gyrus and right supramarginal gyrus in the IGD group. Whole-brain correlational analysis indicated different correlations between the two groups. The brain regions that showed group differences were considered to be involved in cognitive control, decision making, and reward/loss processing. These functions may serve as potential mechanisms that explain why IGD individuals experience negative outcomes in frequent game playing. This article is protected by copyright. All rights reserved. This article is protected by copyright. All rights reserved.
-
Temporal coding of reward-guided choice in the posterior parietal cortex
Hawellek, David J.; Wong, Yan T.; Pesaran, Bijan
2016-01-01
Making a decision involves computations across distributed cortical and subcortical networks. How such distributed processing is performed remains unclear. We test how the encoding of choice in a key decision-making node, the posterior parietal cortex (PPC), depends on the temporal structure of the surrounding population activity. We recorded spiking and local field potential (LFP) activity in the PPC while two rhesus macaques performed a decision-making task. We quantified the mutual information that neurons carried about an upcoming choice and its dependence on LFP activity. The spiking of PPC neurons was correlated with LFP phases at three distinct time scales in the theta, beta, and gamma frequency bands. Importantly, activity at these time scales encoded upcoming decisions differently. Choice information contained in neural firing varied with the phase of beta and gamma activity. For gamma activity, maximum choice information occurred at the same phase as the maximum spike count. However, for beta activity, choice information and spike count were greatest at different phases. In contrast, theta activity did not modulate the encoding properties of PPC units directly but was correlated with beta and gamma activity through cross-frequency coupling. We propose that the relative timing of local spiking and choice information reveals temporal reference frames for computations in either local or large-scale decision networks. Differences between the timing of task information and activity patterns may be a general signature of distributed processing across large-scale networks. PMID:27821752
-
How reward and emotional stimuli induce different reactions across the menstrual cycle
Sakaki, Michiko; Mather, Mara
2012-01-01
Despite widespread belief that moods are affected by the menstrual cycle, researchers on emotion and reward have not paid much attention to the menstrual cycle until recently. However, recent research has revealed different reactions to emotional stimuli and to rewarding stimuli across the different phases of the menstrual cycle. The current paper reviews the emerging literature on how ovarian hormone fluctuation during the menstrual cycle modulates reactions to emotional stimuli and to reward. Behavioral and neuroimaging studies in humans suggest that estrogen and progesterone have opposing influences. That is, it appears that estrogen enhances reactions to reward, but progesterone counters the facilitative effects of estrogen and decreases reactions to rewards. In contrast, reactions to emotionally arousing stimuli (particularly negative stimuli) appear to be decreased by estrogen but enhanced by progesterone. Potential factors that can modulate the effects of the ovarian hormones (e.g., an inverse quadratic function of hormones’ effects; the structural changes of the hippocampus across the menstrual cycle) are also discussed. PMID:22737180
-
Cross-national prevalence and cultural correlates of bipolar I disorder.
Johnson, Kaja R; Johnson, Sheri L
2014-07-01
Bipolar disorder has been consistently related to heightened sensitivity to reward. Greater reward sensitivity predicts the onset of disorder, a more severe course, and conversion from milder to severe forms. No studies consider whether cultural factors related to reward sensitivity influence the course of bipolar disorder. This study examines the relationship of reward-relevant cultural values to global prevalence rates of bipolar I disorder. Lifetime prevalence of bipolar I disorder for 17 countries was drawn from epidemiological studies that used structured diagnostic interviews of large community samples. Bivariate correlations were used to assess the relationship of bipolar disorder prevalence with national scores on four reward-relevant cultural dimensions (Power Distance, Individualism, Long-Term Orientation, and Performance Orientation). The prevalence of bipolar I disorder was correlated in the predicted manner with Power Distance and Individualism, and with Long-Term Orientation and Performance Orientation after outliers were removed. Findings provide evidence for a cultural model of reward sensitivity in bipolar disorder.
-
Compositional clustering in task structure learning
Frank, Michael J.
2018-01-01
Humans are remarkably adept at generalizing knowledge between experiences in a way that can be difficult for computers. Often, this entails generalizing constituent pieces of experiences that do not fully overlap, but nonetheless share useful similarities with, previously acquired knowledge. However, it is often unclear how knowledge gained in one context should generalize to another. Previous computational models and data suggest that rather than learning about each individual context, humans build latent abstract structures and learn to link these structures to arbitrary contexts, facilitating generalization. In these models, task structures that are more popular across contexts are more likely to be revisited in new contexts. However, these models can only re-use policies as a whole and are unable to transfer knowledge about the transition structure of the environment even if only the goal has changed (or vice-versa). This contrasts with ecological settings, where some aspects of task structure, such as the transition function, will be shared between context separately from other aspects, such as the reward function. Here, we develop a novel non-parametric Bayesian agent that forms independent latent clusters for transition and reward functions, affording separable transfer of their constituent parts across contexts. We show that the relative performance of this agent compared to an agent that jointly clusters reward and transition functions depends environmental task statistics: the mutual information between transition and reward functions and the stochasticity of the observations. We formalize our analysis through an information theoretic account of the priors, and propose a meta learning agent that dynamically arbitrates between strategies across task domains to optimize a statistical tradeoff. PMID:29672581
-
Opel, Nils; Redlich, Ronny; Grotegerd, Dominik; Dohm, Katharina; Haupenthal, Cordula; Heindel, Walter; Kugel, Harald; Arolt, Volker; Dannlowski, Udo
2015-06-01
Obesity has been characterized by alterations in brain structure and function associated with emotion processing and regulation. Particularly, aberrations in food-related reward processing have been frequently demonstrated in obese subjects. However, it remains unclear whether reward-associated functional aberrations in obesity are specific for food-related stimuli or represent a general deficit in reward processing, extending to other stimulus domains. Given the crucial role of rewarding effects in the development of obesity and the ongoing discussion on overlapping neurobiological traits of obesity and psychiatric disorders such as depression and substance-related disorders, this study aimed to investigate the possibility of altered reward processing in obese subjects to occur in the absence of food-related stimuli during a monetary reward condition. Twenty-nine healthy obese subjects (body mass index >30) and 29 healthy, age-, and sex-matched control subjects of normal weight underwent functional MRI during a frequently used card guessing paradigm. A Group × Condition (win vs. loss) ANOVA was conducted to investigate differences between obese and normal-weight subjects. We found significant Group × Condition interaction effects in brain areas involved in emotion regulation and reward processing including the insula, the striatum, and the orbitofrontal cortex (OFC). This interaction was predominantly driven by a significant increase in blood oxygenation level dependent (BOLD) response in obese individuals while experiencing reward. Enhanced neural activation in obesity during reward processing seems to be apparent even in the absence of food-related stimuli and, thus, might point to generalized dysfunctions in reward-related brain circuits in obese individuals. © 2015 Wiley Periodicals, Inc.
-
Risk, Reward, and Regulations: Reshaping the Financial Services Industry.
ERIC Educational Resources Information Center
Saul, Ralph S.
1984-01-01
Deregulation has had much to do with the competitive drive and vitality of financial services within the United States. The risks and rewards of deregulation for financial institutions are discussed, and principles which should serve as a guide in building any new regulatory structure are examined. (RM)
-
Institutionalizing Faculty Mentoring within a Community of Practice Model
ERIC Educational Resources Information Center
Smith, Emily R.; Calderwood, Patricia E.; Storms, Stephanie Burrell; Lopez, Paula Gill; Colwell, Ryan P.
2016-01-01
In higher education, faculty work is typically enacted--and rewarded--on an individual basis. Efforts to promote collaboration run counter to the individual and competitive reward systems that characterize higher education. Mentoring initiatives that promote faculty collaboration and support also defy the structural and cultural norms of higher…
-
Performance Pay for Teachers: Determinants and Consequences
ERIC Educational Resources Information Center
Belfield, Clive R.; Heywood, John S.
2008-01-01
Theory and evidence on performance-related pay for teaching remain inconclusive. Teachers will respond to rewards, but an appropriate reward structure may not be devised because education is a collaborative endeavor. Here we test three hypotheses: performance-related pay among teachers is more likely to be observed when there are evident…
-
Jokisch, Daniel; Roser, Patrik; Juckel, Georg; Daum, Irene; Bellebaum, Christian
2014-07-01
Excessive alcohol consumption has been linked to structural and functional brain changes associated with cognitive, emotional, and behavioral impairments. It has been suggested that neural processing in the reward system is also affected by alcoholism. The present study aimed at further investigating reward-based associative learning and reversal learning in detoxified alcohol-dependent patients. Twenty-one detoxified alcohol-dependent patients and 26 healthy control subjects participated in a probabilistic learning task using monetary and alcohol-associated rewards as feedback stimuli indicating correct responses. Performance during acquisition and reversal learning in the different feedback conditions was analyzed. Alcohol-dependent patients and healthy control subjects showed an increase in learning performance over learning blocks during acquisition, with learning performance being significantly lower in alcohol-dependent patients. After changing the contingencies, alcohol-dependent patients exhibited impaired reversal learning and showed, in contrast to healthy controls, different learning curves for different types of rewards with no increase in performance for high monetary and alcohol-associated feedback. The present findings provide evidence that dysfunctional processing in the reward system in alcohol-dependent patients leads to alterations in reward-based learning resulting in a generally reduced performance. In addition, the results suggest that alcohol-dependent patients are, in particular, more impaired in changing an established behavior originally reinforced by high rewards. Copyright © 2014 by the Research Society on Alcoholism.
-
Dissociating movement from movement timing in the rat primary motor cortex.
Knudsen, Eric B; Powers, Marissa E; Moxon, Karen A
2014-11-19
Neural encoding of the passage of time to produce temporally precise movements remains an open question. Neurons in several brain regions across different experimental contexts encode estimates of temporal intervals by scaling their activity in proportion to the interval duration. In motor cortex the degree to which this scaled activity relies upon afferent feedback and is guided by motor output remains unclear. Using a neural reward paradigm to dissociate neural activity from motor output before and after complete spinal transection, we show that temporally scaled activity occurs in the rat hindlimb motor cortex in the absence of motor output and after transection. Context-dependent changes in the encoding are plastic, reversible, and re-established following injury. Therefore, in the absence of motor output and despite a loss of afferent feedback, thought necessary for timed movements, the rat motor cortex displays scaled activity during a broad range of temporally demanding tasks similar to that identified in other brain regions. Copyright © 2014 the authors 0270-6474/14/3415576-11$15.00/0.
-
Sridharan, Vishnupriya; Cohen, Trevor; Cobb, Nathan; Myneni, Sahiti
2016-01-01
With online social platforms gaining popularity as venues of behavior change, it is important to understand the ways in which these platforms facilitate peer interactions. In this paper, we characterize temporal trends in user communication through mapping of theoretically-linked semantic content. We used qualitative coding and automated text analysis to assign theoretical techniques to peer interactions in an online community for smoking cessation, subsequently facilitating temporal visualization of the observed techniques. Results indicate manifestation of several behavior change techniques such as feedback and monitoring' and 'rewards'. Automated methods yielded reasonable results (F-measure=0.77). Temporal trends among relapsers revealed reduction in communication after a relapse event. This social withdrawal may be attributed to failure guilt after the relapse. Results indicate significant change in thematic categories such as 'social support', 'natural consequences', and 'comparison of outcomes' pre and post relapse. Implications for development of behavioral support technologies that promote long-term abstinence are discussed.
-
'Wanting' and 'liking' skin picking: A validation of the Skin Picking Reward Scale.
Snorrason, Ivar; Olafsson, Ragnar P; Houghton, David C; Woods, Douglas W; Lee, Han-Joo
2015-12-01
Excoriation (skin-picking) disorder (SPD) is often conceptualized as a behavioral addiction in which aberrant reward processing may play an important role. The current study sought to develop a self-report instrument--the Skin Picking Reward Scale (SPRS)--that measures how strongly skin picking is 'liked' (i.e., the degree of pleasurable feelings while receiving the reward) and 'wanted' (i.e., the degree of the motivation to seek the reward). We administered the SPRS to individuals who endorsed excessive skin picking in online surveys and examined the scale's factor structure (Studies 1 and 2). We then asked individuals with documented pathological skin picking to complete the SPRS and other relevant questionnaires on two occasions one week apart (Study 3). Exploratory (Study 1; n = 330) and confirmatory (Study 2; n = 144) factor analyses consistently supported a two-factor structure reflecting the 'liking' and 'wanting' constructs. Results from Study 3 (N = 36) indicated that the Wanting and the Liking scales had adequate internal consistency and test-retest reliability. Additionally, consistent with predictions, the Wanting scale, but not the Liking scale, was associated with picking urges the following week, greater cue-reactivity, and more picking-related routines/habits. These initial findings suggest that SPRS is a psychometrically sound measure of 'wanting' and 'liking' in pathological skin picking. The SPRS may facilitate research on reward processing anomalies in SPD and serve as a useful clinical instrument (e.g., to identify those at risk for cue-induced relapse).
-
de Jonge, Jan; van der Linden, Sjaak; Schaufeli, Wilmar; Peter, Richard; Siegrist, Johannes
2008-01-01
Key measures of Siegrist's (1996) Effort-Reward Imbalance (ERI) Model (i.e., efforts, rewards, and overcommitment) were psychometrically tested. To study change in organizational interventions, knowledge about the type of change underlying the instruments used is needed. Next to assessing baseline factorial validity and reliability, the factorial stability over time - known as alpha-beta-gamma change - of the ERI scales was examined. Psychometrics were tested among 383 and 267 healthcare workers from two Dutch panel surveys with different time lags. Baseline results favored a five-factor model (i.e., efforts, esteem rewards, financial/career-related aspects, job security, and overcommitment) over and above a three-factor solution (i.e., efforts, composite rewards, and overcommitment). Considering changes as a whole, particularly the factor loadings of the three ERI scales were not equal over time. Findings suggest in general that moderate changes in the ERI factor structure did not affect the interpretation of mean changes over time. Occupational health researchers utilizing the ERI scales can feel confident that self-reported changes are more likely to be due to factors other than structural change of the ERI scales over time, which has important implications for evaluating job stress and health interventions.
-
Hsu, Chia-Fen; Benikos, Nicholas; Sonuga-Barke, Edmund J S
2015-04-01
Spontaneous very low frequency oscillations (VLFO), seen in the resting brain, are attenuated when individuals are working on attention demanding tasks or waiting for rewards (Hsu et al., 2013). Individuals with attention-deficit/hyperactivity disorder (ADHD) display excess VLFO when working on attention tasks. They also have difficulty waiting for rewards. Here we examined the waiting brain signature in ADHD and its association with impulsive choice. DC-EEG from 21 children with ADHD and 21 controls (9-15 years) were collected under four conditions: (i) resting; (ii) choosing to wait; (iii) being "forced" to wait; and (iv) working on a reaction time task. A questionnaire measured two components of impulsive choice. Significant VLFO reductions were observed in controls within anterior brain regions in both working and waiting conditions. Individuals with ADHD showed VLFO attenuation while working but to a reduced level and none at all when waiting. A closer inspection revealed an increase of VLFO activity in temporal regions during waiting. Excess VLFO activity during waiting was associated with parents' ratings of temporal discounting and delay aversion. The results highlight the potential role for waiting-related spontaneous neural activity in the pathophysiology of impulsive decision-making of ADHD. Copyright © 2015 The Authors. Published by Elsevier Ltd.. All rights reserved.
-
fMRI study of neural sensitization to hedonic stimuli in long-term, daily cannabis users.
Filbey, Francesca M; Dunlop, Joseph; Ketcherside, Ariel; Baine, Jessica; Rhinehardt, Tyler; Kuhn, Brittany; DeWitt, Sam; Alvi, Talha
2016-10-01
Although there is emergent evidence illustrating neural sensitivity to cannabis cues in cannabis users, the specificity of this effect to cannabis cues as opposed to a generalized hyper-sensitivity to hedonic stimuli has not yet been directly tested. Using fMRI, we presented 53 daily, long-term cannabis users and 68 non-using controls visual and tactile cues for cannabis, a natural reward, and, a sensory-perceptual control object to evaluate brain response to hedonic stimuli in cannabis users. The results showed an interaction between group and reward type such that the users had greater response during cannabis cues relative to natural reward cues (i.e., fruit) in the orbitofrontal cortex, striatum, anterior cingulate gyrus, and ventral tegmental area compared to non-users (cluster-threshold z = 2.3, P < 0.05). In the users, there were positive brain-behavior correlations between neural response to cannabis cues in fronto-striatal-temporal regions and subjective craving, marijuana-related problems, withdrawal symptoms, and levels of THC metabolites (cluster-threshold z = 2.3, P < 0.05). These findings demonstrate hyper-responsivity, and, specificity of brain response to cannabis cues in long-term cannabis users that are above that of response to natural reward cues. These observations are concordant with incentive sensitization models suggesting sensitization of mesocorticolimbic regions and disruption of natural reward processes following drug use. Although the cross-sectional nature of this study does not provide information on causality, the positive correlations between neural response and indicators of cannabis use (i.e., THC levels) suggest that alterations in the reward system are, in part, related to cannabis use. Hum Brain Mapp 37:3431-3443, 2016. © 2016 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc. © 2016 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc.
-
Reward-Guided Learning with and without Causal Attribution
Jocham, Gerhard; Brodersen, Kay H.; Constantinescu, Alexandra O.; Kahn, Martin C.; Ianni, Angela M.; Walton, Mark E.; Rushworth, Matthew F.S.; Behrens, Timothy E.J.
2016-01-01
Summary When an organism receives a reward, it is crucial to know which of many candidate actions caused this reward. However, recent work suggests that learning is possible even when this most fundamental assumption is not met. We used novel reward-guided learning paradigms in two fMRI studies to show that humans deploy separable learning mechanisms that operate in parallel. While behavior was dominated by precise contingent learning, it also revealed hallmarks of noncontingent learning strategies. These learning mechanisms were separable behaviorally and neurally. Lateral orbitofrontal cortex supported contingent learning and reflected contingencies between outcomes and their causal choices. Amygdala responses around reward times related to statistical patterns of learning. Time-based heuristic mechanisms were related to activity in sensorimotor corticostriatal circuitry. Our data point to the existence of several learning mechanisms in the human brain, of which only one relies on applying known rules about the causal structure of the task. PMID:26971947
-
Journal Impact Factor Shapes Scientists’ Reward Signal in the Prospect of Publication
Paulus, Frieder Michel; Rademacher, Lena; Schäfer, Theo Alexander Jose; Müller-Pinzler, Laura; Krach, Sören
2015-01-01
The incentive structure of a scientist’s life is increasingly mimicking economic principles. While intensely criticized, the journal impact factor (JIF) has taken a role as the new currency for scientists. Successful goal-directed behavior in academia thus requires knowledge about the JIF. Using functional neuroimaging we examined how the JIF, as a powerful incentive in academia, has shaped the behavior of scientists and the reward signal in the striatum. We demonstrate that the reward signal in the nucleus accumbens increases with higher JIF during the anticipation of a publication and found a positive correlation with the personal publication record (pJIF) supporting the notion that scientists have incorporated the predominant reward principle of the scientific community in their reward system. The implications of this behavioral adaptation within the ecological niche of the scientist’s habitat remain unknown, but may also have effects which were not intended by the community. PMID:26555725
-
Mentalizing and motivation neural function during social interactions in autism spectrum disorders☆
Assaf, Michal; Hyatt, Christopher J.; Wong, Christina G.; Johnson, Matthew R.; Schultz, Robert T.; Hendler, Talma; Pearlson, Godfrey D.
2013-01-01
Autism Spectrum Disorders (ASDs) are characterized by core deficits in social functions. Two theories have been suggested to explain these deficits: mind-blindness theory posits impaired mentalizing processes (i.e. decreased ability for establishing a representation of others' state of mind), while social motivation theory proposes that diminished reward value for social information leads to reduced social attention, social interactions, and social learning. Mentalizing and motivation are integral to typical social interactions, and neuroimaging evidence points to independent brain networks that support these processes in healthy individuals. However, the simultaneous function of these networks has not been explored in individuals with ASDs. We used a social, interactive fMRI task, the Domino game, to explore mentalizing- and motivation-related brain activation during a well-defined interval where participants respond to rewards or punishments (i.e. motivation) and concurrently process information about their opponent's potential next actions (i.e. mentalizing). Thirteen individuals with high-functioning ASDs, ages 12–24, and 14 healthy controls played fMRI Domino games against a computer-opponent and separately, what they were led to believe was a human-opponent. Results showed that while individuals with ASDs understood the game rules and played similarly to controls, they showed diminished neural activity during the human-opponent runs only (i.e. in a social context) in bilateral middle temporal gyrus (MTG) during mentalizing and right Nucleus Accumbens (NAcc) during reward-related motivation (Pcluster < 0.05 FWE). Importantly, deficits were not observed in these areas when playing against a computer-opponent or in areas related to motor and visual processes. These results demonstrate that while MTG and NAcc, which are critical structures in the mentalizing and motivation networks, respectively, activate normally in a non-social context, they fail to respond in an otherwise identical social context in ASD compared to controls. We discuss implications to both the mind-blindness and social motivation theories of ASD and the importance of social context in research and treatment protocols. PMID:24273716
-
Music and the nucleus accumbens.
Mavridis, Ioannis N
2015-03-01
Music is a universal feature of human societies over time, mainly because it allows expression and regulation of strong emotions, thus influencing moods and evoking pleasure. The nucleus accumbens (NA), the most important pleasure center of the human brain (dominates the reward system), is the 'king of neurosciences' and dopamine (DA) can be rightfully considered as its 'crown' due to the fundamental role that this neurotransmitter plays in the brain's reward system. Purpose of this article was to review the existing literature regarding the relation between music and the NA. Studies have shown that reward value for music can be coded by activity levels in the NA, whose functional connectivity with auditory and frontal areas increases as a function of increasing musical reward. Listening to music strongly modulates activity in a network of mesolimbic structures involved in reward processing including the NA. The functional connectivity between brain regions mediating reward, autonomic and cognitive processing provides insight into understanding why listening to music is one of the most rewarding and pleasurable human experiences. Musical stimuli can significantly increase extracellular DA levels in the NA. NA DA and serotonin were found significantly higher in animals exposed to music. Finally, passive listening to unfamiliar although liked music showed activations in the NA.
-
Caudate clues to rewarding cues.
Platt, Michael L
2002-01-31
Behavioral studies indicate that prior experience can influence discrimination of subsequent stimuli. The mechanisms responsible for highlighting a particular aspect of the stimulus, such as motion or color, as most relevant and thus deserving further scrutiny, however, remain poorly understood. In the current issue of Neuron, demonstrate that neurons in the caudate nucleus of the basal ganglia signal which dimension of a visual cue, either color or location, is associated with reward in an eye movement task. These findings raise the possibility that this structure participates in the reward-based control of visual attention.
-
Experience Corps Baltimore: Exploring the Stressors and Rewards of High-intensity Civic Engagement
Varma, Vijay R.; Carlson, Michelle C.; Parisi, Jeanine M.; Tanner, Elizabeth K.; McGill, Sylvia; Fried, Linda P.; Song, Linda H.; Gruenewald, Tara L.
2015-01-01
Purpose: Experience Corps (EC) represents a high-intensity, intergenerational civic engagement activity where older adults serve as mentors and tutors in elementary schools. Although high-intensity volunteer opportunities are designed to enhance the health and well being of older adult volunteers, little is known about the negative and positive aspects of volunteering unique to intergenerational programs from the volunteer’s perspective. Design and Methods: Stressors and rewards associated with volunteering in EC were explored in 8 focus group discussions with 46 volunteers from EC Baltimore. Transcripts were coded for frequently expressed themes. Results: Participants reported stressors and rewards within 5 key domains: intergenerational (children’s problem behavior, working with and helping children, observing/facilitating improvement or transformation in a child, and developing a special connection with a child); external to EC (poor parenting and children’s social stressors); interpersonal (challenges in working with teachers and bonding/making social connections); personal (enjoyment, self-enhancement/achievement, and being/feeling more active); and structural (satisfaction with the structural elements of the EC program). Implications: Volunteers experienced unique intergenerational stressors related to children’s problem behavior and societal factors external to the EC program. Overall, intergenerational, interpersonal, and personal rewards from volunteering, as well as program structure may have balanced the stress associated with volunteering. A better understanding of stressors and rewards from high-intensity volunteer programs may enhance our understanding of how intergenerational civic engagement volunteering affects well being in later life and may inform project modifications to maximize such benefits for future volunteers and those they serve. PMID:24589989
-
Neither Honor nor Compensation: Faculty and Public Service
ERIC Educational Resources Information Center
Jaeger, Audrey J.; Thornton, Courtney H.
2006-01-01
Land-grant institutions that are also research extensive may face the unique position of asking faculty to fulfill a historical mission with low consideration of public service in the reward structure. This paradox between mission and reward results in the socialization of many faculty away from participation in public service. This article…
-
Altbäcker, Anna; Plózer, Enikő; Darnai, Gergely; Perlaki, Gábor; Horváth, Réka; Orsi, Gergely; Nagy, Szilvia Anett; Bogner, Péter; Schwarcz, Attila; Kovács, Norbert; Komoly, Sámuel; Clemens, Zsófia; Janszky, József
2016-12-01
Neuroimaging findings suggest that excessive Internet use shows functional and structural brain changes similar to substance addiction. Even though it is still under debate whether there are gender differences in case of problematic use, previous studies by-passed this question by focusing on males only or by using gender matched approach without controlling for potential gender effects. We designed our study to find out whether there are structural correlates in the brain reward system of problematic Internet use in habitual Internet user females. T1-weighted Magnetic Resonance (MR) images were collected in 82 healthy habitual Internet user females. Structural brain measures were investigated using both automated MR volumetry and voxel based morphometry (VBM). Self-reported measures of problematic Internet use and hours spent online were also assessed. According to MR volumetry, problematic Internet use was associated with increased grey matter volume of bilateral putamen and right nucleus accumbens while decreased grey matter volume of orbitofrontal cortex (OFC). Similarly, VBM analysis revealed a significant negative association between the absolute amount of grey matter OFC and problematic Internet use. Our findings suggest structural brain alterations in the reward system usually related to addictions are present in problematic Internet use.
-
Policy improvement by a model-free Dyna architecture.
Hwang, Kao-Shing; Lo, Chia-Yue
2013-05-01
The objective of this paper is to accelerate the process of policy improvement in reinforcement learning. The proposed Dyna-style system combines two learning schemes, one of which utilizes a temporal difference method for direct learning; the other uses relative values for indirect learning in planning between two successive direct learning cycles. Instead of establishing a complicated world model, the approach introduces a simple predictor of average rewards to actor-critic architecture in the simulation (planning) mode. The relative value of a state, defined as the accumulated differences between immediate reward and average reward, is used to steer the improvement process in the right direction. The proposed learning scheme is applied to control a pendulum system for tracking a desired trajectory to demonstrate its adaptability and robustness. Through reinforcement signals from the environment, the system takes the appropriate action to drive an unknown dynamic to track desired outputs in few learning cycles. Comparisons are made between the proposed model-free method, a connectionist adaptive heuristic critic, and an advanced method of Dyna-Q learning in the experiments of labyrinth exploration. The proposed method outperforms its counterparts in terms of elapsed time and convergence rate.
-
Rapuano, Kristina M.; Huckins, Jeremy F.; Sargent, James D.; Heatherton, Todd F.; Kelley, William M.
2016-01-01
The prevalence of adolescent obesity has increased dramatically over the past three decades, and research has documented that the number of television shows viewed during childhood is associated with greater risk for obesity. In particular, considerable evidence suggests that exposure to food marketing promotes eating habits that contribute to obesity. The present study examines neural responses to dynamic food commercials in overweight and healthy-weight adolescents using functional magnetic resonance imaging (fMRI). Compared with non-food commercials, food commercials more strongly engaged regions involved in attention and saliency detection (occipital lobe, precuneus, superior temporal gyri, and right insula) and in processing rewards [left and right nucleus accumbens (NAcc) and left orbitofrontal cortex (OFC)]. Activity in the left OFC and right insula further correlated with subjects' percent body fat at the time of the scan. Interestingly, this reward-related activity to food commercials was accompanied by the additional recruitment of mouth-specific somatosensory-motor cortices—a finding that suggests the intriguing possibility that higher-adiposity adolescents mentally simulate eating behaviors and offers a potential neural mechanism for the formation and reinforcement of unhealthy eating habits that may hamper an individual's ability lose weight later in life. PMID:25994961
-
Cahill, Michael E.; Bagot, Rosemary C.; Gancarz, Amy M.; Walker, Deena M.; Sun, HaoSheng; Wang, Zi-Jun; Heller, Elizabeth A.; Feng, Jian; Kennedy, Pamela J.; Koo, Ja Wook; Cates, Hannah M.; Neve, Rachael L.; Shen, Li; Dietz, David M.
2016-01-01
Summary Dendritic spines are the sites of most excitatory synapses in the CNS, and opposing alterations in the synaptic structure of medium spiny neurons (MSNs) of the nucleus accumbens, a primary brain reward region, are seen at early vs. late time points after cocaine administration. Here we investigate the time-dependent molecular and biochemical processes that regulate this bidirectional synaptic structural plasticity of NAc MSNs and associated changes in cocaine reward in response to chronic cocaine exposure. Our findings reveal key roles for the bidirectional synaptic expression of the Rap1b small GTPase and an associated local-synaptic protein translation network in this process. The transcriptional mechanisms and pathway-specific inputs to NAc that regulate Rap1b expression are also characterized. Collectively, these findings provide a precise mechanism by which nuclear to synaptic interactions induce “metaplasticity” in NAc MSNs, and we reveal the specific effects of this plasticity on reward behavior in a brain circuit-specific manner. PMID:26844834
-
Novelty Enhances Visual Salience Independently of Reward in the Parietal Lobe
Foley, Nicholas C.; Jangraw, David C.; Peck, Christopher
2014-01-01
Novelty modulates sensory and reward processes, but it remains unknown how these effects interact, i.e., how the visual effects of novelty are related to its motivational effects. A widespread hypothesis, based on findings that novelty activates reward-related structures, is that all the effects of novelty are explained in terms of reward. According to this idea, a novel stimulus is by default assigned high reward value and hence high salience, but this salience rapidly decreases if the stimulus signals a negative outcome. Here we show that, contrary to this idea, novelty affects visual salience in the monkey lateral intraparietal area (LIP) in ways that are independent of expected reward. Monkeys viewed peripheral visual cues that were novel or familiar (received few or many exposures) and predicted whether the trial will have a positive or a negative outcome—i.e., end in a reward or a lack of reward. We used a saccade-based assay to detect whether the cues automatically attracted or repelled attention from their visual field location. We show that salience—measured in saccades and LIP responses—was enhanced by both novelty and positive reward associations, but these factors were dissociable and habituated on different timescales. The monkeys rapidly recognized that a novel stimulus signaled a negative outcome (and withheld anticipatory licking within the first few presentations), but the salience of that stimulus remained high for multiple subsequent presentations. Therefore, novelty can provide an intrinsic bonus for attention that extends beyond the first presentation and is independent of physical rewards. PMID:24899716
-
Stress and reward processing in bipolar disorder: an fMRI study
Berghorst, Lisa H; Kumar, Poornima; Greve, Doug N; Deckersbach, Thilo; Ongur, Dost; Dutra, Sunny; Pizzagalli, Diego A
2016-01-01
Objectives A link between negative life stress and the onset of mood episodes in bipolar disorder (BD) has been established, but processes underlying such a link remain unclear. Growing evidence suggests that stress can negatively affect reward processing and related neurobiological substrates, indicating that a dysregulated reward system may provide a partial explanation. The aim of this study was to test the impact of stress on reward-related neural functioning in BD. Methods Thirteen euthymic or mildly depressed individuals with BD and 15 controls performed a Monetary Incentive Delay task while undergoing functional magnetic resonance imaging during no-stress and stress (negative psychosocial stressor involving poor performance feedback and threat of monetary deductions) conditions. Results In hypothesis-driven region-of- interest-based analyses, a significant group by condition interaction emerged in the amygdala during reward anticipation. Relative to controls, while anticipating a potential reward, subjects with BD were characterized by amygdalar hyperactivation in the no-stress condition but hypoactivation during stress. Moreover, relative to controls, subjects with BD had significantly larger amygdala volumes. After controlling for structural differences, the effects of stress on amygdalar function remained, whereas groups no longer differed during the no-stress condition. During reward consumption, a group by condition interaction emerged in the putamen due to increased putamen activation to rewards in participants with BD during stress, but an opposite pattern in controls. Conclusions Overall, findings highlight possible impairments in using reward-predicting cues to adaptively engage in goal-directed actions in BD, combined with stress-induced hypersensitivity to reward consumption. Potential clinical implications are discussed. PMID:27870507
-
Novelty enhances visual salience independently of reward in the parietal lobe.
Foley, Nicholas C; Jangraw, David C; Peck, Christopher; Gottlieb, Jacqueline
2014-06-04
Novelty modulates sensory and reward processes, but it remains unknown how these effects interact, i.e., how the visual effects of novelty are related to its motivational effects. A widespread hypothesis, based on findings that novelty activates reward-related structures, is that all the effects of novelty are explained in terms of reward. According to this idea, a novel stimulus is by default assigned high reward value and hence high salience, but this salience rapidly decreases if the stimulus signals a negative outcome. Here we show that, contrary to this idea, novelty affects visual salience in the monkey lateral intraparietal area (LIP) in ways that are independent of expected reward. Monkeys viewed peripheral visual cues that were novel or familiar (received few or many exposures) and predicted whether the trial will have a positive or a negative outcome--i.e., end in a reward or a lack of reward. We used a saccade-based assay to detect whether the cues automatically attracted or repelled attention from their visual field location. We show that salience--measured in saccades and LIP responses--was enhanced by both novelty and positive reward associations, but these factors were dissociable and habituated on different timescales. The monkeys rapidly recognized that a novel stimulus signaled a negative outcome (and withheld anticipatory licking within the first few presentations), but the salience of that stimulus remained high for multiple subsequent presentations. Therefore, novelty can provide an intrinsic bonus for attention that extends beyond the first presentation and is independent of physical rewards. Copyright © 2014 the authors 0270-6474/14/347947-11$15.00/0.
-
Time to rethink the neural mechanisms of learning and memory
Gallistel, Charles R.; Balsam, Peter D
2014-01-01
Most studies in the neurobiology of learning assume that the underlying learning process is a pairing – dependent change in synaptic strength that requires repeated experience of events presented in close temporal contiguity. However, much learning is rapid and does not depend on temporal contiguity which has never been precisely defined. These points are well illustrated by studies showing that temporal relationships between events are rapidly learned-even over long delays- and this knowledge governs the form and timing of behavior. The speed with which anticipatory responses emerge in conditioning paradigms is determined by the information that cues provide about the timing of rewards. The challenge for understanding the neurobiology of learning is to understand the mechanisms in the nervous system that encode information from even a single experience, the nature of the memory mechanisms that can encode quantities such as time, and how the brain can flexibly perform computations based on this information. PMID:24309167
-
Abnormal Social Reward Responses in Anorexia Nervosa: An fMRI Study.
Via, Esther; Soriano-Mas, Carles; Sánchez, Isabel; Forcano, Laura; Harrison, Ben J; Davey, Christopher G; Pujol, Jesús; Martínez-Zalacaín, Ignacio; Menchón, José M; Fernández-Aranda, Fernando; Cardoner, Narcís
2015-01-01
Patients with anorexia nervosa (AN) display impaired social interactions, implicated in the development and prognosis of the disorder. Importantly, social behavior is modulated by reward-based processes, and dysfunctional at-brain-level reward responses have been involved in AN neurobiological models. However, no prior evidence exists of whether these neural alterations would be equally present in social contexts. In this study, we conducted a cross-sectional social-judgment functional magnetic resonance imaging (fMRI) study of 20 restrictive-subtype AN patients and 20 matched healthy controls. Brain activity during acceptance and rejection was investigated and correlated with severity measures (Eating Disorder Inventory -EDI-2) and with personality traits of interest known to modulate social behavior (The Sensitivity to Punishment and Sensitivity to Reward Questionnaire). Patients showed hypoactivation of the dorsomedial prefrontal cortex (DMPFC) during social acceptance and hyperactivation of visual areas during social rejection. Ventral striatum activation during rejection was positively correlated in patients with clinical severity scores. During acceptance, activation of the frontal opercula-anterior insula and dorsomedial/dorsolateral prefrontal cortices was differentially associated with reward sensitivity between groups. These results suggest an abnormal motivational drive for social stimuli, and involve overlapping social cognition and reward systems leading to a disruption of adaptive responses in the processing of social reward. The specific association of reward-related regions with clinical and psychometric measures suggests the putative involvement of reward structures in the maintenance of pathological behaviors in AN.
-
The role of reward in word learning and its implications for language acquisition.
Ripollés, Pablo; Marco-Pallarés, Josep; Hielscher, Ulrike; Mestres-Missé, Anna; Tempelmann, Claus; Heinze, Hans-Jochen; Rodríguez-Fornells, Antoni; Noesselt, Toemme
2014-11-03
The exact neural processes behind humans' drive to acquire a new language--first as infants and later as second-language learners--are yet to be established. Recent theoretical models have proposed that during human evolution, emerging language-learning mechanisms might have been glued to phylogenetically older subcortical reward systems, reinforcing human motivation to learn a new language. Supporting this hypothesis, our results showed that adult participants exhibited robust fMRI activation in the ventral striatum (VS)--a core region of reward processing--when successfully learning the meaning of new words. This activation was similar to the VS recruitment elicited using an independent reward task. Moreover, the VS showed enhanced functional and structural connectivity with neocortical language areas during successful word learning. Together, our results provide evidence for the neural substrate of reward and motivation during word learning. We suggest that this strong functional and anatomical coupling between neocortical language regions and the subcortical reward system provided a crucial advantage in humans that eventually enabled our lineage to successfully acquire linguistic skills. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
McAuley, J Devin; Henry, Molly J; Wedd, Alan; Pleskac, Timothy J; Cesario, Joseph
2012-02-01
Two experiments investigated the effects of musicality and motivational orientation on auditory category learning. In both experiments, participants learned to classify tone stimuli that varied in frequency and duration according to an initially unknown disjunctive rule; feedback involved gaining points for correct responses (a gains reward structure) or losing points for incorrect responses (a losses reward structure). For Experiment 1, participants were told at the start that musicians typically outperform nonmusicians on the task, and then they were asked to identify themselves as either a "musician" or a "nonmusician." For Experiment 2, participants were given either a promotion focus prime (a performance-based opportunity to gain entry into a raffle) or a prevention focus prime (a performance-based criterion that needed to be maintained to avoid losing an entry into a raffle) at the start of the experiment. Consistent with a regulatory-fit hypothesis, self-identified musicians and promotion-primed participants given a gains reward structure made more correct tone classifications and were more likely to discover the optimal disjunctive rule than were musicians and promotion-primed participants experiencing losses. Reward structure (gains vs. losses) had inconsistent effects on the performance of nonmusicians, and a weaker regulatory-fit effect was found for the prevention focus prime. Overall, the findings from this study demonstrate a regulatory-fit effect in the domain of auditory category learning and show that motivational orientation may contribute to musician performance advantages in auditory perception.
-
Do gamblers eat more salt? Testing a latent trait model of covariance in consumption
Goodwin, Belinda C.; Browne, Matthew; Rockloff, Matthew; Donaldson, Phillip
2015-01-01
A diverse class of stimuli, including certain foods, substances, media, and economic behaviours, may be described as ‘reward-oriented’ in that they provide immediate reinforcement with little initial investment. Neurophysiological and personality concepts, including dopaminergic dysfunction, reward sensitivity and rash impulsivity, each predict the existence of a latent behavioural trait that leads to increased consumption of all stimuli in this class. Whilst bivariate relationships (co-morbidities) are often reported in the literature, to our knowledge, a multivariate investigation of this possible trait has not been done. We surveyed 1,194 participants (550 male) on their typical weekly consumption of 11 types of reward-oriented stimuli, including fast food, salt, caffeine, television, gambling products, and illicit drugs. Confirmatory factor analysis was used to compare models in a 3×3 structure, based on the definition of a single latent factor (none, fixed loadings, or estimated loadings), and assumed residual covariance structure (none, a-priori / literature based, or post-hoc / data-driven). The inclusion of a single latent behavioural ‘consumption’ factor significantly improved model fit in all cases. Also confirming theoretical predictions, estimated factor loadings on reward-oriented indicators were uniformly positive, regardless of assumptions regarding residual covariances. Additionally, the latent trait was found to be negatively correlated with the non-reward-oriented indicators of fruit and vegetable consumption. The findings support the notion of a single behavioural trait leading to increased consumption of reward-oriented stimuli across multiple modalities. We discuss implications regarding the concentration of negative lifestyle-related health behaviours. PMID:26551907
-
Do gamblers eat more salt? Testing a latent trait model of covariance in consumption.
Goodwin, Belinda C; Browne, Matthew; Rockloff, Matthew; Donaldson, Phillip
2015-09-01
A diverse class of stimuli, including certain foods, substances, media, and economic behaviours, may be described as 'reward-oriented' in that they provide immediate reinforcement with little initial investment. Neurophysiological and personality concepts, including dopaminergic dysfunction, reward sensitivity and rash impulsivity, each predict the existence of a latent behavioural trait that leads to increased consumption of all stimuli in this class. Whilst bivariate relationships (co-morbidities) are often reported in the literature, to our knowledge, a multivariate investigation of this possible trait has not been done. We surveyed 1,194 participants (550 male) on their typical weekly consumption of 11 types of reward-oriented stimuli, including fast food, salt, caffeine, television, gambling products, and illicit drugs. Confirmatory factor analysis was used to compare models in a 3×3 structure, based on the definition of a single latent factor (none, fixed loadings, or estimated loadings), and assumed residual covariance structure (none, a-priori / literature based, or post-hoc / data-driven). The inclusion of a single latent behavioural 'consumption' factor significantly improved model fit in all cases. Also confirming theoretical predictions, estimated factor loadings on reward-oriented indicators were uniformly positive, regardless of assumptions regarding residual covariances. Additionally, the latent trait was found to be negatively correlated with the non-reward-oriented indicators of fruit and vegetable consumption. The findings support the notion of a single behavioural trait leading to increased consumption of reward-oriented stimuli across multiple modalities. We discuss implications regarding the concentration of negative lifestyle-related health behaviours.
-
Rutledge, Robb B.; Zaehle, Tino; Schmitt, Friedhelm C.; Kopitzki, Klaus; Kowski, Alexander B.; Voges, Jürgen; Heinze, Hans-Jochen; Dolan, Raymond J.
2015-01-01
Functional magnetic resonance imaging (fMRI), cyclic voltammetry, and single-unit electrophysiology studies suggest that signals measured in the nucleus accumbens (Nacc) during value-based decision making represent reward prediction errors (RPEs), the difference between actual and predicted rewards. Here, we studied the precise temporal and spectral pattern of reward-related signals in the human Nacc. We recorded local field potentials (LFPs) from the Nacc of six epilepsy patients during an economic decision-making task. On each trial, patients decided whether to accept or reject a gamble with equal probabilities of a monetary gain or loss. The behavior of four patients was consistent with choices being guided by value expectations. Expected value signals before outcome onset were observed in three of those patients, at varying latencies and with nonoverlapping spectral patterns. Signals after outcome onset were correlated with RPE regressors in all subjects. However, further analysis revealed that these signals were better explained as outcome valence rather than RPE signals, with gamble gains and losses differing in the power of beta oscillations and in evoked response amplitudes. Taken together, our results do not support the idea that postsynaptic potentials in the Nacc represent a RPE that unifies outcome magnitude and prior value expectation. We discuss the generalizability of our findings to healthy individuals and the relation of our results to measurements of RPE signals obtained from the Nacc with other methods. PMID:26019312
-
Stenner, Max-Philipp; Rutledge, Robb B; Zaehle, Tino; Schmitt, Friedhelm C; Kopitzki, Klaus; Kowski, Alexander B; Voges, Jürgen; Heinze, Hans-Jochen; Dolan, Raymond J
2015-08-01
Functional magnetic resonance imaging (fMRI), cyclic voltammetry, and single-unit electrophysiology studies suggest that signals measured in the nucleus accumbens (Nacc) during value-based decision making represent reward prediction errors (RPEs), the difference between actual and predicted rewards. Here, we studied the precise temporal and spectral pattern of reward-related signals in the human Nacc. We recorded local field potentials (LFPs) from the Nacc of six epilepsy patients during an economic decision-making task. On each trial, patients decided whether to accept or reject a gamble with equal probabilities of a monetary gain or loss. The behavior of four patients was consistent with choices being guided by value expectations. Expected value signals before outcome onset were observed in three of those patients, at varying latencies and with nonoverlapping spectral patterns. Signals after outcome onset were correlated with RPE regressors in all subjects. However, further analysis revealed that these signals were better explained as outcome valence rather than RPE signals, with gamble gains and losses differing in the power of beta oscillations and in evoked response amplitudes. Taken together, our results do not support the idea that postsynaptic potentials in the Nacc represent a RPE that unifies outcome magnitude and prior value expectation. We discuss the generalizability of our findings to healthy individuals and the relation of our results to measurements of RPE signals obtained from the Nacc with other methods. Copyright © 2015 the American Physiological Society.
-
Rewards and advancements for clinical pharmacists.
Goodwin, S Diane; Kane-Gill, Sandra L; Ng, Tien M H; Melroy, Joel T; Hess, Mary M; Tallian, Kimberly; Trujillo, Toby C; Vermeulen, Lee C
2010-01-01
The American College of Clinical Pharmacy charged the Clinical Practice Affairs Committee to review and update the College's 1995 White Paper, "Rewards and Advancements for Clinical Pharmacy Practitioners." Because of the limited data on the present state of rewards and advancements for clinical pharmacists, an online survey of "front-line" clinical pharmacists and pharmacy managers was conducted (1126 total respondents, 14% response rate). The resulting White Paper discusses motivators and existing systems of rewards and advancements for clinical pharmacists, as well as perceived barriers to implementation of these systems. Clinical pharmacists reported work-life balance, a challenging position, and opportunities for professional advancement as the most important factors for career success. At the time of the survey, financial rewards appeared not to be a major motivator for clinical pharmacists. Managers underestimated the importance that clinical pharmacists place on work-life balance and favorable work schedules. Although almost two thirds of the clinical pharmacists surveyed had not developed a professional development plan, 84% indicated an interest in career planning. Both clinical pharmacists and managers rated the lack of a clear reward and advancement structure as the most important barrier to effective systems of rewards and advancements. Pharmacy managers and administrators are encouraged to develop effective systems of rewards and advancements for clinical pharmacists that positively impact patient care and the institution's mission; these systems will benefit the clinical pharmacist, the health care institution, and the patient.
-
Prosocial Reward Learning in Children and Adolescents
Kwak, Youngbin; Huettel, Scott A.
2016-01-01
Adolescence is a period of increased sensitivity to social contexts. To evaluate how social context sensitivity changes over development—and influences reward learning—we investigated how children and adolescents perceive and integrate rewards for oneself and others during a dynamic risky decision-making task. Children and adolescents (N = 75, 8–16 years) performed the Social Gambling Task (SGT, Kwak et al., 2014) and completed a set of questionnaires measuring other-regarding behavior. In the SGT, participants choose amongst four card decks that have different payout structures for oneself and for a charity. We examined patterns of choices, overall decision strategies, and how reward outcomes led to trial-by-trial adjustments in behavior, as estimated using a reinforcement-learning model. Performance of children and adolescents was compared to data from a previously collected sample of adults (N = 102) performing the identical task. We found that that children/adolescents were not only more sensitive to rewards directed to the charity than self but also showed greater prosocial tendencies on independent measures of other-regarding behavior. Children and adolescents also showed less use of a strategy that prioritizes rewards for self at the expense of rewards for others. These results support the conclusion that, compared to adults, children and adolescents show greater sensitivity to outcomes for others when making decisions and learning about potential rewards. PMID:27761125
-
Rewarding Community-Engaged Scholarship: A State University System Approach
ERIC Educational Resources Information Center
Saltmarsh, John; Wooding, John
2016-01-01
The need for new and revised structures to reward new forms of scholarship is being examined nationally and globally. It is also being examined on campuses that make up the University of Massachusetts system, all which are classified by the Carnegie Foundation for Community Engagement. This paper reports on the collective exploration by the five…
-
Li, Anan; Gire, David H.
2015-01-01
Studies in different sensory systems indicate that short spike patterns within a spike train that carry items of sensory information can be extracted from the overall train by using field potential oscillations as a reference (Kayser et al., 2012; Panzeri et al., 2014). Here we test the hypothesis that the local field potential (LFP) provides the temporal reference frame needed to differentiate between odors regardless of associated outcome. Experiments were performed in the olfactory system of the mouse (Mus musculus) where the mitral/tufted (M/T) cell spike rate develops differential responses to rewarded and unrewarded odors as the animal learns to associate one of the odors with a reward in a go–no go behavioral task. We found that coherence of spiking in M/T cells with the ϒ LFP (65 to 95 Hz) differentiates between odors regardless of the associated behavioral outcome of odor presentation. PMID:25855190
-
What changed during the axial age: Cognitive styles or reward systems?
Baumard, Nicolas; Hyafil, Alexandre; Boyer, Pascal
2015-01-01
The ‘Axial Age’ (500–300 BCE) refers to the period during which most of the main religious and spiritual traditions emerged in Eurasian societies. Although the Axial Age has recently been the focus of increasing interest,1-5 its existence is still very much in dispute. The main reason for questioning the existence of the Axial Age is that its nature, as well as its spatial and temporal boundaries, remain very much unclear. The standard approach to the Axial Age defines it as a change of cognitive style, from a narrative and analogical style to a more analytical and reflective style, probably due to the increasing use of external memory tools. Our recent research suggests an alternative hypothesis, namely a change in reward orientation, from a short-term materialistic orientation to a long-term spiritual one.6 Here, we briefly discuss these 2 alternative definitions of the Axial Age. PMID:27066164
-
Spike-Timing of Orbitofrontal Neurons Is Synchronized With Breathing.
Kőszeghy, Áron; Lasztóczi, Bálint; Forro, Thomas; Klausberger, Thomas
2018-01-01
The orbitofrontal cortex (OFC) has been implicated in a multiplicity of complex brain functions, including representations of expected outcome properties, post-decision confidence, momentary food-reward values, complex flavors and odors. As breathing rhythm has an influence on odor processing at primary olfactory areas, we tested the hypothesis that it may also influence neuronal activity in the OFC, a prefrontal area involved also in higher order processing of odors. We recorded spike timing of orbitofrontal neurons as well as local field potentials (LFPs) in awake, head-fixed mice, together with the breathing rhythm. We observed that a large majority of orbitofrontal neurons showed robust phase-coupling to breathing during immobility and running. The phase coupling of action potentials to breathing was significantly stronger in orbitofrontal neurons compared to cells in the medial prefrontal cortex. The characteristic synchronization of orbitofrontal neurons with breathing might provide a temporal framework for multi-variable processing of olfactory, gustatory and reward-value relationships.
-
A common currency for the computation of motivational values in the human striatum
Li, Yansong; Dreher, Jean-Claude
2015-01-01
Reward comparison in the brain is thought to be achieved through the use of a ‘common currency’, implying that reward value representations are computed on a unique scale in the same brain regions regardless of the reward type. Although such a mechanism has been identified in the ventro-medial prefrontal cortex and ventral striatum in the context of decision-making, it is less clear whether it similarly applies to non-choice situations. To answer this question, we scanned 38 participants with fMRI while they were presented with single cues predicting either monetary or erotic rewards, without the need to make a decision. The ventral striatum was the main brain structure to respond to both cues while showing increasing activity with increasing expected reward intensity. Most importantly, the relative response of the striatum to monetary vs erotic cues was correlated with the relative motivational value of these rewards as inferred from reaction times. Similar correlations were observed in a fronto-parietal network known to be involved in attentional focus and motor readiness. Together, our results suggest that striatal reward value signals not only obey to a common currency mechanism in the absence of choice but may also serve as an input to adjust motivated behaviour accordingly. PMID:24837478
-
Social Reward Questionnaire—Adolescent Version and its association with callous–unemotional traits
Neumann, Craig S.; Roberts, Ruth; McCrory, Eamon; Viding, Essi
2017-01-01
During adolescence, social interactions are a potent source of reward. However, no measure of social reward value exists for this age group. In this study, we adapted the adult Social Reward Questionnaire, which we had previously developed and validated, for use with adolescents. Participants aged 11–16 (n = 568; 50% male) completed the Social Reward Questionnaire—Adolescent Version (SRQ-A), alongside measures of personality traits—five-factor model (FFM) and callous–unemotional (CU) traits—for construct validity purposes. A confirmatory factor analysis of the SRQ-A supported a five-factor structure (Comparative Fit Index = 0.90; Root Mean Square Error of Approximation = 0.07), equating to five questionnaire subscales: enjoyment of Admiration, Negative Social Potency, Passivity, Prosocial Interactions and Sociability. Associations with FFM and CU traits were in line with what is seen for adult samples, providing support for the meaning of SRQ-A subscales in adolescents. In particular, adolescents with high levels of CU traits showed an ‘inverted’ pattern of social reward, in which being cruel is enjoyable and being kind is not. Gender invariance was also assessed and was partially supported. The SRQ-A is a valid, reliable measure of individual differences in social reward in adolescents. PMID:28484617
-
NASA Astrophysics Data System (ADS)
Dabanlı, İsmail; Şen, Zekai
2018-04-01
The statistical climate downscaling model by the Turkish Water Foundation (TWF) is further developed and applied to a set of monthly precipitation records. The model is structured by two phases as spatial (regional) and temporal downscaling of global circulation model (GCM) scenarios. The TWF model takes into consideration the regional dependence function (RDF) for spatial structure and Markov whitening process (MWP) for temporal characteristics of the records to set projections. The impact of climate change on monthly precipitations is studied by downscaling Intergovernmental Panel on Climate Change-Special Report on Emission Scenarios (IPCC-SRES) A2 and B2 emission scenarios from Max Plank Institute (EH40PYC) and Hadley Center (HadCM3). The main purposes are to explain the TWF statistical climate downscaling model procedures and to expose the validation tests, which are rewarded in same specifications as "very good" for all stations except one (Suhut) station in the Akarcay basin that is in the west central part of Turkey. Eventhough, the validation score is just a bit lower at the Suhut station, the results are "satisfactory." It is, therefore, possible to say that the TWF model has reasonably acceptable skill for highly accurate estimation regarding standard deviation ratio (SDR), Nash-Sutcliffe efficiency (NSE), and percent bias (PBIAS) criteria. Based on the validated model, precipitation predictions are generated from 2011 to 2100 by using 30-year reference observation period (1981-2010). Precipitation arithmetic average and standard deviation have less than 5% error for EH40PYC and HadCM3 SRES (A2 and B2) scenarios.
-
Bayesian parameter estimation for nonlinear modelling of biological pathways.
Ghasemi, Omid; Lindsey, Merry L; Yang, Tianyi; Nguyen, Nguyen; Huang, Yufei; Jin, Yu-Fang
2011-01-01
The availability of temporal measurements on biological experiments has significantly promoted research areas in systems biology. To gain insight into the interaction and regulation of biological systems, mathematical frameworks such as ordinary differential equations have been widely applied to model biological pathways and interpret the temporal data. Hill equations are the preferred formats to represent the reaction rate in differential equation frameworks, due to their simple structures and their capabilities for easy fitting to saturated experimental measurements. However, Hill equations are highly nonlinearly parameterized functions, and parameters in these functions cannot be measured easily. Additionally, because of its high nonlinearity, adaptive parameter estimation algorithms developed for linear parameterized differential equations cannot be applied. Therefore, parameter estimation in nonlinearly parameterized differential equation models for biological pathways is both challenging and rewarding. In this study, we propose a Bayesian parameter estimation algorithm to estimate parameters in nonlinear mathematical models for biological pathways using time series data. We used the Runge-Kutta method to transform differential equations to difference equations assuming a known structure of the differential equations. This transformation allowed us to generate predictions dependent on previous states and to apply a Bayesian approach, namely, the Markov chain Monte Carlo (MCMC) method. We applied this approach to the biological pathways involved in the left ventricle (LV) response to myocardial infarction (MI) and verified our algorithm by estimating two parameters in a Hill equation embedded in the nonlinear model. We further evaluated our estimation performance with different parameter settings and signal to noise ratios. Our results demonstrated the effectiveness of the algorithm for both linearly and nonlinearly parameterized dynamic systems. Our proposed Bayesian algorithm successfully estimated parameters in nonlinear mathematical models for biological pathways. This method can be further extended to high order systems and thus provides a useful tool to analyze biological dynamics and extract information using temporal data.
-
Stress and reward processing in bipolar disorder: a functional magnetic resonance imaging study.
Berghorst, Lisa H; Kumar, Poornima; Greve, Doug N; Deckersbach, Thilo; Ongur, Dost; Dutra, Sunny J; Pizzagalli, Diego A
2016-11-01
A link between negative life stress and the onset of mood episodes in bipolar disorder (BD) has been established, but processes underlying such a link remain unclear. Growing evidence suggests that stress can negatively affect reward processing and related neurobiological substrates, indicating that a dysregulated reward system may provide a partial explanation. The aim of this study was to test the impact of stress on reward-related neural functioning in BD. Thirteen euthymic or mildly depressed individuals with BD and 15 controls performed a Monetary Incentive Delay (MID) task while undergoing functional magnetic resonance imaging during no-stress and stress (negative psychosocial stressor involving poor performance feedback and threat of monetary deductions) conditions. In hypothesis-driven region-of-interest analyses, a significant group-by-condition interaction emerged in the amygdala during reward anticipation. Relative to controls, while anticipating a potential reward, subjects with BD were characterized by amygdalar hyperactivation in the no-stress condition but hypoactivation during stress. Moreover, relative to controls, subjects with BD had significantly larger amygdala volumes. After controlling for structural differences, the effects of stress on amygdalar function remained, whereas groups no longer differed during the no-stress condition. During reward consumption, a group-by-condition interaction emerged in the putamen due to increased putamen activation in response to rewards in participants with BD during stress, but an opposite pattern in controls. Overall, findings highlight possible impairments in using reward-predicting cues to adaptively engage in goal-directed actions in BD, combined with stress-induced hypersensitivity to reward consumption. Potential clinical implications are discussed. © 2016 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.
-
Ross, Shani E; Lehmann Levin, Emily; Itoga, Christy A; Schoen, Chelsea B; Selmane, Romeissa; Aldridge, J Wayne
2016-10-01
We investigated the potential of deep brain stimulation (DBS) in the central nucleus of the amygdala (CeA) in rats to modulate functional reward mechanisms. The CeA is the major output of the amygdala with direct connections to the hypothalamus and gustatory brainstem, and indirect connections with the nucleus accumbens. Further, the CeA has been shown to be involved in learning, emotional integration, reward processing, and regulation of feeding. We hypothesized that DBS, which is used to treat movement disorders and other brain dysfunctions, might block reward motivation. In rats performing a lever-pressing task to obtain sugar pellet rewards, we stimulated the CeA and control structures, and compared stimulation parameters. During CeA stimulation, animals stopped working for rewards and rejected freely available rewards. Taste reactivity testing during DBS exposed aversive reactions to normally liked sucrose tastes and even more aversive taste reactions to normally disliked quinine tastes. Interestingly, given the opportunity, animals implanted in the CeA would self-stimulate with 500 ms trains of stimulation at the same frequency and current parameters as continuous stimulation that would stop reward acquisition. Neural recordings during DBS showed that CeA neurons were still active and uncovered inhibitory-excitatory patterns after each stimulus pulse indicating possible entrainment of the neural firing with DBS. In summary, DBS modulation of CeA may effectively usurp normal neural activity patterns to create an 'information lesion' that not only decreased motivational 'wanting' of food rewards, but also blocked 'liking' of rewards. © 2016 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.
-
Experience Corps Baltimore: Exploring the Stressors and Rewards of High-intensity Civic Engagement.
Varma, Vijay R; Carlson, Michelle C; Parisi, Jeanine M; Tanner, Elizabeth K; McGill, Sylvia; Fried, Linda P; Song, Linda H; Gruenewald, Tara L
2015-12-01
Experience Corps (EC) represents a high-intensity, intergenerational civic engagement activity where older adults serve as mentors and tutors in elementary schools. Although high-intensity volunteer opportunities are designed to enhance the health and well being of older adult volunteers, little is known about the negative and positive aspects of volunteering unique to intergenerational programs from the volunteer's perspective. Stressors and rewards associated with volunteering in EC were explored in 8 focus group discussions with 46 volunteers from EC Baltimore. Transcripts were coded for frequently expressed themes. Participants reported stressors and rewards within 5 key domains: intergenerational (children's problem behavior, working with and helping children, observing/facilitating improvement or transformation in a child, and developing a special connection with a child); external to EC (poor parenting and children's social stressors); interpersonal (challenges in working with teachers and bonding/making social connections); personal (enjoyment, self-enhancement/achievement, and being/feeling more active); and structural (satisfaction with the structural elements of the EC program). Volunteers experienced unique intergenerational stressors related to children's problem behavior and societal factors external to the EC program. Overall, intergenerational, interpersonal, and personal rewards from volunteering, as well as program structure may have balanced the stress associated with volunteering. A better understanding of stressors and rewards from high-intensity volunteer programs may enhance our understanding of how intergenerational civic engagement volunteering affects well being in later life and may inform project modifications to maximize such benefits for future volunteers and those they serve. © The Author 2014. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
-
Vaquero, Lucía; Cámara, Estela; Sampedro, Frederic; Pérez de Los Cobos, José; Batlle, Francesca; Fabregas, Josep Maria; Sales, Joan Artur; Cervantes, Mercè; Ferrer, Xavier; Lazcano, Gerardo; Rodríguez-Fornells, Antoni; Riba, Jordi
2017-05-01
Cocaine addiction has been associated with increased sensitivity of the human reward circuit to drug-related stimuli. However, the capacity of non-drug incentives to engage this network is poorly understood. Here, we characterized the functional sensitivity to monetary incentives and the structural integrity of the human reward circuit in abstinent cocaine-dependent (CD) patients and their matched controls. We assessed the BOLD response to monetary gains and losses in 30 CD patients and 30 healthy controls performing a lottery task in a magnetic resonance imaging scanner. We measured brain gray matter volume (GMV) using voxel-based morphometry and white matter microstructure using voxel-based fractional anisotropy (FA). Functional data showed that, after monetary incentives, CD patients exhibited higher activation in the ventral striatum than controls. Furthermore, we observed an inverted BOLD response pattern in the prefrontal cortex, with activity being highest after unexpected high gains and lowest after losses. Patients showed increased GMV in the caudate and the orbitofrontal cortex, increased white matter FA in the orbito-striatal pathway but decreased FA in antero-posterior association bundles. Abnormal activation in the prefrontal cortex correlated with GMV and FA increases in the orbitofrontal cortex. While functional abnormalities in the ventral striatum were inversely correlated with abstinence duration, structural alterations were not. In conclusion, results suggest abnormal incentive processing in CD patients with high salience for rewards and punishments in subcortical structures but diminished prefrontal control after adverse outcomes. They further suggest that hypertrophy and hyper-connectivity within the reward circuit, to the expense of connectivity outside this network, characterize cocaine addiction. © 2016 Society for the Study of Addiction.
-
Planning activity for internally generated reward goals in monkey amygdala neurons
Schultz, Wolfram
2015-01-01
The best rewards are often distant and can only be achieved by planning and decision-making over several steps. We designed a multi-step choice task in which monkeys followed internal plans to save rewards towards self-defined goals. During this self-controlled behavior, amygdala neurons showed future-oriented activity that reflected the animal’s plan to obtain specific rewards several trials ahead. This prospective activity encoded crucial components of the animal’s plan, including value and length of the planned choice sequence. It began on initial trials when a plan would be formed, reappeared step-by-step until reward receipt, and readily updated with a new sequence. It predicted performance, including errors, and typically disappeared during instructed behavior. Such prospective activity could underlie the formation and pursuit of internal plans characteristic for goal-directed behavior. The existence of neuronal planning activity in the amygdala suggests an important role for this structure in guiding behavior towards internally generated, distant goals. PMID:25622146
-
Two spatiotemporally distinct value systems shape reward-based learning in the human brain.
Fouragnan, Elsa; Retzler, Chris; Mullinger, Karen; Philiastides, Marios G
2015-09-08
Avoiding repeated mistakes and learning to reinforce rewarding decisions is critical for human survival and adaptive actions. Yet, the neural underpinnings of the value systems that encode different decision-outcomes remain elusive. Here coupling single-trial electroencephalography with simultaneously acquired functional magnetic resonance imaging, we uncover the spatiotemporal dynamics of two separate but interacting value systems encoding decision-outcomes. Consistent with a role in regulating alertness and switching behaviours, an early system is activated only by negative outcomes and engages arousal-related and motor-preparatory brain structures. Consistent with a role in reward-based learning, a later system differentially suppresses or activates regions of the human reward network in response to negative and positive outcomes, respectively. Following negative outcomes, the early system interacts and downregulates the late system, through a thalamic interaction with the ventral striatum. Critically, the strength of this coupling predicts participants' switching behaviour and avoidance learning, directly implicating the thalamostriatal pathway in reward-based learning.
-
Sign Tracking, but Not Goal Tracking, is Resistant to Outcome Devaluation
Morrison, Sara E.; Bamkole, Michael A.; Nicola, Saleem M.
2015-01-01
During Pavlovian conditioning, a conditioned stimulus (CS) may act as a predictor of a reward to be delivered in another location. Individuals vary widely in their propensity to engage with the CS (sign tracking) or with the site of eventual reward (goal tracking). It is often assumed that sign tracking involves the association of the CS with the motivational value of the reward, resulting in the CS acquiring incentive value independent of the outcome. However, experimental evidence for this assumption is lacking. In order to test the hypothesis that sign tracking behavior does not rely on a neural representation of the outcome, we employed a reward devaluation procedure. We trained rats on a classic Pavlovian paradigm in which a lever CS was paired with a sucrose reward, then devalued the reward by pairing sucrose with illness in the absence of the CS. We found that sign tracking behavior was enhanced, rather than diminished, following reward devaluation; thus, sign tracking is clearly independent of a representation of the outcome. In contrast, goal tracking behavior was decreased by reward devaluation. Furthermore, when we divided rats into those with high propensity to engage with the lever (sign trackers) and low propensity to engage with the lever (goal trackers), we found that nearly all of the effects of devaluation could be attributed to the goal trackers. These results show that sign tracking and goal tracking behavior may be the output of different associative structures in the brain, providing insight into the mechanisms by which reward-associated stimuli—such as drug cues—come to exert control over behavior in some individuals. PMID:26733783
-
Narcissism is associated with weakened frontostriatal connectivity: a DTI study
Lynam, Donald R.; Powell, David K.; DeWall, C. Nathan
2016-01-01
Narcissism is characterized by the search for affirmation and admiration from others. Might this motivation to find external sources of acclaim exist to compensate for neurostructural deficits that link the self with reward? Greater structural connectivity between brain areas that process self-relevant stimuli (i.e. the medial prefrontal cortex) and reward (i.e. the ventral striatum) is associated with fundamentally positive self-views. We predicted that narcissism would be associated with less integrity of this frontostriatal pathway. We used diffusion tensor imaging to assess the frontostriatal structural connectivity among 50 healthy undergraduates (32 females, 18 males) who also completed a measure of grandiose narcissism. White matter integrity in the frontostriatal pathway was negatively associated with narcissism. Our findings, while purely correlational, suggest that narcissism arises, in part, from a neural disconnect between the self and reward. The exhibitionism and immodesty of narcissists may then be a regulatory strategy to compensate for this neural deficit. PMID:26048178
-
Sitek, Kevin R.; Cai, Shanqing; Beal, Deryk S.; Perkell, Joseph S.; Guenther, Frank H.; Ghosh, Satrajit S.
2016-01-01
Persistent developmental stuttering is characterized by speech production disfluency and affects 1% of adults. The degree of impairment varies widely across individuals and the neural mechanisms underlying the disorder and this variability remain poorly understood. Here we elucidate compensatory mechanisms related to this variability in impairment using whole-brain functional and white matter connectivity analyses in persistent developmental stuttering. We found that people who stutter had stronger functional connectivity between cerebellum and thalamus than people with fluent speech, while stutterers with the least severe symptoms had greater functional connectivity between left cerebellum and left orbitofrontal cortex (OFC). Additionally, people who stutter had decreased functional and white matter connectivity among the perisylvian auditory, motor, and speech planning regions compared to typical speakers, but greater functional connectivity between the right basal ganglia and bilateral temporal auditory regions. Structurally, disfluency ratings were negatively correlated with white matter connections to left perisylvian regions and to the brain stem. Overall, we found increased connectivity among subcortical and reward network structures in people who stutter compared to controls. These connections were negatively correlated with stuttering severity, suggesting the involvement of cerebellum and OFC may underlie successful compensatory mechanisms by more fluent stutterers. PMID:27199712
-
Floral rewards in the tribe Sisyrinchieae (Iridaceae): oil as an alternative to pollen and nectar?
Silvério, Adriano; Nadot, Sophie; Souza-Chies, Tatiana T; Chauveau, Olivier
2012-12-01
Iridaceae is one of the few families in which floral oils are produced and collected by pollinators as a resource. Perigonal nectaries and trichomal elaiophores are highly unusual within the tribe Sisyrinchieae. Both structures occur mainly on the staminal column, while they are usually distributed on the tepals in the other tribes of the subfamily Iridoideae. Sisyrinchieae is the largest tribe of Iridaceae present on the American continent, and the diversity observed may be related to the exceptional development of trichomal elaiophores within the genus Sisyrinchium, but knowledge concerning the other types of nuptial glandular structures within the tribe is still limited, preventing us from estimating their implication for species diversity. Structural observations and histochemical tests were performed to identify and characterize glandular structures and pollen rewards within the flowers of the genera Orthrosanthus, Sisyrinchium and Solenomelus. Perigonal nectaries were detected only in Solenomelus segethi, and trichomal elaiophores were characterized only within Sisyrinchium. All species showed large amounts of additional resources available for pollinators in the form of pollenkitt and polysaccharides present in the cytoplasm of the pollen grains. The results are discussed in a phylogenetic context, with regard to pollinators and floral rewards reported for the tribe Sisyrinchieae.
-
Observed Parenting Behavior with Teens: Measurement Invariance and Predictive Validity Across Race
Skinner, Martie L.; MacKenzie, Elizabeth P.; Haggerty, Kevin P.; Hill, Karl G.; Roberson, Kendra C.
2011-01-01
Previous reports supporting measurement equality between European American and African American families have often focused on self-reported risk factors or observed parent behavior with young children. This study examines equality of measurement of observer ratings of parenting behavior with adolescents during structured tasks; mean levels of observed parenting; and predictive validity of teen self-reports of antisocial behaviors and beliefs using a sample of 163 African American and 168 European American families. Multiple-group confirmatory factor analyses supported measurement invariance across ethnic groups for 4 measures of observed parenting behavior: prosocial rewards, psychological costs, antisocial rewards, and problem solving. Some mean-level differences were found: African American parents exhibited lower levels of prosocial rewards, higher levels of psychological costs, and lower problem solving when compared to European Americans. No significant mean difference was found in rewards for antisocial behavior. Multigroup structural equation models suggested comparable relationships across race (predictive validity) between parenting constructs and youth antisocial constructs (i.e., drug initiation, positive drug attitudes, antisocial attitudes, problem behaviors) in all but one of the tested relationships. This study adds to existing evidence that family-based interventions targeting parenting behaviors can be generalized to African American families. PMID:21787057
-
Stimulus-Response-Outcome Coding in the Pigeon Nidopallium Caudolaterale
Starosta, Sarah; Güntürkün, Onur; Stüttgen, Maik C.
2013-01-01
A prerequisite for adaptive goal-directed behavior is that animals constantly evaluate action outcomes and relate them to both their antecedent behavior and to stimuli predictive of reward or non-reward. Here, we investigate whether single neurons in the avian nidopallium caudolaterale (NCL), a multimodal associative forebrain structure and a presumed analogue of mammalian prefrontal cortex, represent information useful for goal-directed behavior. We subjected pigeons to a go-nogo task, in which responding to one visual stimulus (S+) was partially reinforced, responding to another stimulus (S–) was punished, and responding to test stimuli from the same physical dimension (spatial frequency) was inconsequential. The birds responded most intensely to S+, and their response rates decreased monotonically as stimuli became progressively dissimilar to S+; thereby, response rates provided a behavioral index of reward expectancy. We found that many NCL neurons' responses were modulated in the stimulus discrimination phase, the outcome phase, or both. A substantial fraction of neurons increased firing for cues predicting non-reward or decreased firing for cues predicting reward. Interestingly, the same neurons also responded when reward was expected but not delivered, and could thus provide a negative reward prediction error or, alternatively, signal negative value. In addition, many cells showed motor-related response modulation. In summary, NCL neurons represent information about the reward value of specific stimuli, instrumental actions as well as action outcomes, and therefore provide signals useful for adaptive behavior in dynamically changing environments. PMID:23437383
-
Relation of obesity to neural activation in response to food commercials
Yokum, Sonja; Stice, Eric; Harris, Jennifer L.; Brownell, Kelly D.
2014-01-01
Adolescents view thousands of food commercials annually, but the neural response to food advertising and its association with obesity is largely unknown. This study is the first to examine how neural response to food commercials differs from other stimuli (e.g. non-food commercials and television show) and to explore how this response may differ by weight status. The blood oxygen level-dependent functional magnetic resonance imaging activation was measured in 30 adolescents ranging from lean to obese in response to food and non-food commercials imbedded in a television show. Adolescents exhibited greater activation in regions implicated in visual processing (e.g. occipital gyrus), attention (e.g. parietal lobes), cognition (e.g. temporal gyrus and posterior cerebellar lobe), movement (e.g. anterior cerebellar cortex), somatosensory response (e.g. postcentral gyrus) and reward [e.g. orbitofrontal cortex and anterior cingulate cortex (ACC)] during food commercials. Obese participants exhibited less activation during food relative to non-food commercials in neural regions implicated in visual processing (e.g. cuneus), attention (e.g. posterior cerebellar lobe), reward (e.g. ventromedial prefrontal cortex and ACC) and salience detection (e.g. precuneus). Obese participants did exhibit greater activation in a region implicated in semantic control (e.g. medial temporal gyrus). These findings may inform current policy debates regarding the impact of food advertising to minors. PMID:23576811
-
Brain reward circuitry beyond the mesolimbic dopamine system: a neurobiological theory.
Ikemoto, Satoshi
2010-11-01
Reductionist attempts to dissect complex mechanisms into simpler elements are necessary, but not sufficient for understanding how biological properties like reward emerge out of neuronal activity. Recent studies on intracranial self-administration of neurochemicals (drugs) found that rats learn to self-administer various drugs into the mesolimbic dopamine structures-the posterior ventral tegmental area, medial shell nucleus accumbens and medial olfactory tubercle. In addition, studies found roles of non-dopaminergic mechanisms of the supramammillary, rostromedial tegmental and midbrain raphe nuclei in reward. To explain intracranial self-administration and related effects of various drug manipulations, I outlined a neurobiological theory claiming that there is an intrinsic central process that coordinates various selective functions (including perceptual, visceral, and reinforcement processes) into a global function of approach. Further, this coordinating process for approach arises from interactions between brain structures including those structures mentioned above and their closely linked regions: the medial prefrontal cortex, septal area, ventral pallidum, bed nucleus of stria terminalis, preoptic area, lateral hypothalamic areas, lateral habenula, periaqueductal gray, laterodorsal tegmental nucleus and parabrachical area. Published by Elsevier Ltd.
-
A θ-γ oscillation code for neuronal coordination during motor behavior.
Igarashi, Jun; Isomura, Yoshikazu; Arai, Kensuke; Harukuni, Rie; Fukai, Tomoki
2013-11-20
Sequential motor behavior requires a progression of discrete preparation and execution states. However, the organization of state-dependent activity in neuronal ensembles of motor cortex is poorly understood. Here, we recorded neuronal spiking and local field potential activity from rat motor cortex during reward-motivated movement and observed robust behavioral state-dependent coordination between neuronal spiking, γ oscillations, and θ oscillations. Slow and fast γ oscillations appeared during distinct movement states and entrained neuronal firing. γ oscillations, in turn, were coupled to θ oscillations, and neurons encoding different behavioral states fired at distinct phases of θ in a highly layer-dependent manner. These findings indicate that θ and nested dual band γ oscillations serve as the temporal structure for the selection of a conserved set of functional channels in motor cortical layer activity during animal movement. Furthermore, these results also suggest that cross-frequency couplings between oscillatory neuronal ensemble activities are part of the general coding mechanism in cortex.
-
Temporal framing and the hidden-zero effect: rate-dependent outcomes on delay discounting.
Naudé, Gideon P; Kaplan, Brent A; Reed, Derek D; Henley, Amy J; DiGennaro Reed, Florence D
2018-05-01
Recent research suggests that presenting time intervals as units (e.g., days) or as specific dates, can modulate the degree to which humans discount delayed outcomes. Another framing effect involves explicitly stating that choosing a smaller-sooner reward is mutually exclusive to receiving a larger-later reward, thus presenting choices as an extended sequence. In Experiment 1, participants (N = 201) recruited from Amazon Mechanical Turk completed the Monetary Choice Questionnaire in a 2 (delay framing) by 2 (zero framing) design. Regression suggested a main effect of delay, but not zero, framing after accounting for other demographic variables and manipulations. We observed a rate-dependent effect for the date-framing group, such that those with initially steep discounting exhibited greater sensitivity to the manipulation than those with initially shallow discounting. Subsequent analyses suggest these effects cannot be explained by regression to the mean. Experiment 2 addressed the possibility that the null effect of zero framing was due to within-subject exposure to the hidden- and explicit-zero conditions. A new Amazon Mechanical Turk sample completed the Monetary Choice Questionnaire in either hidden- or explicit-zero formats. Analyses revealed a main effect of reward magnitude, but not zero framing, suggesting potential limitations to the generality of the hidden-zero effect. © 2018 Society for the Experimental Analysis of Behavior.
-
Ely, Alice V; Childress, Anna Rose; Jagannathan, Kanchana; Lowe, Michael R
2015-12-01
Normal weight historical dieters (HDs) are prone to future weight gain, and show higher levels of brain activation in reward-related regions after having eaten than nondieters (NDs) in response to food stimuli (Ely, Childress, Jagannathan, & Lowe, 2014), a similar pattern to that seen in obesity. We hypothesized that HDs are differentially sensitive after eating to rewards in general, and thus extended prior findings by comparing the same groups' brain activation when viewing romantic pictures compared to neutral stimuli while being scanned in a blood oxygenation level-dependent (BOLD) fMRI paradigm in a fasted and fed state. Results show that 1) in fed relative to fasted conditions, both HDs and NDs were more responsive in areas related to reward and 2) in HDs, greater fed versus fasted activation extended to areas linked to perception and goal-directed behavior. HDs relative to NDs were more responsive to romantic cues in the superior frontal gyrus when fasted and the middle temporal gyrus when fed. This pattern of response is similar to HDs' activation when viewing highly palatable food cues, and is consistent with research showing overlapping brain-based responses to sex, drugs and food. Copyright © 2015 Elsevier Ltd. All rights reserved.
-
Social interaction recruits mentalizing and reward systems in middle childhood.
Alkire, Diana; Levitas, Daniel; Warnell, Katherine Rice; Redcay, Elizabeth
2018-06-08
Social cognition develops in the context of reciprocal social interaction. However, most neuroimaging studies of mentalizing have used noninteractive tasks that may fail to capture important aspects of real-world mentalizing. In adults, social-interactive context modulates activity in regions linked to social cognition and reward, but few interactive studies have been done with children. The current fMRI study examines children aged 8-12 using a novel paradigm in which children believed they were interacting online with a peer. We compared mental and non-mental state reasoning about a live partner (Peer) versus a story character (Character), testing the effects of mentalizing and social interaction in a 2 × 2 design. Mental versus Non-Mental reasoning engaged regions identified in prior mentalizing studies, including the temporoparietal junction, superior temporal sulcus, and dorsomedial prefrontal cortex. Moreover, peer interaction, even in conditions without explicit mentalizing demands, activated many of the same mentalizing regions. Peer interaction also activated areas outside the traditional mentalizing network, including the reward system. Our results demonstrate that social interaction engages multiple neural systems during middle childhood and contribute further evidence that social-interactive paradigms are needed to fully capture how the brain supports social processing in the real world. © 2018 Wiley Periodicals, Inc.
-
Can Decision Making Research Provide a Better Understanding of Chemical and Behavioral Addictions?
Engel, Anzhelika; Cáceda, Ricardo
2015-01-01
We reviewed the cognitive and neurobiological commonalities between chemical and behavioral addictions. Poor impulse control, limited executive function and abnormalities in reward processing are seen in both group of entities. Brain imaging shows consistent abnormalities in frontoparietal regions and the limbic system. In drug addiction, exaggerated risk taking behavior and temporal discounting may reflect an imbalance between a hyperactive mesolimbic and hypoactive executive systems. Several cognitive distortions are found in pathological gambling that seems to harness the brain reward system that has evolved to face situations related to skill, not random chance. Abnormalities in risk assessment and impulsivity are found in variety of eating disorders, in particularly related to eating behavior. Corresponding findings in eating disorder patients include abnormalities in the limbic system, i.e. orbitofrontal cortex (OFC), striatum and insula. Similarly, internet addiction disorder is associated with risky decision making and increased choice impulsivity with corresponding discrepant activation in the dorsolateral prefrontal cortex, OFC, anterior cingulate cortex, caudate and insula. Sexual addictions are in turn associated with exaggerated impulsive choice and suggestive evidence of abnormalities in reward processing. In sum, exploration of executive function and decision making abnormalities in chemical and behavioral addictions may increase understanding in their psychopathology and yield valuable targets for therapeutic interventions.
-
Weissman, David G; Schriber, Roberta A; Fassbender, Catherine; Atherton, Olivia; Krafft, Cynthia; Robins, Richard W; Hastings, Paul D; Guyer, Amanda E
2015-12-01
Early adolescent onset of substance use is a robust predictor of future substance use disorders. We examined the relation between age of substance use initiation and resting state functional connectivity (RSFC) of the core reward processing (nucleus accumbens; NAcc) to cognitive control (prefrontal cortex; PFC) brain networks. Adolescents in a longitudinal study of Mexican-origin youth reported their substance use annually from ages 10 to 16 years. At age 16, 69 adolescents participated in a resting state functional magnetic resonance imaging scan. Seed-based correlational analyses were conducted using regions of interest in bilateral NAcc. The earlier that adolescents initiated substance use, the stronger the connectivity between bilateral NAcc and right dorsolateral PFC, right dorsomedial PFC, right pre-supplementary motor area, right inferior parietal lobule, and left medial temporal gyrus. The regions that demonstrated significant positive linear relationships between the number of adolescent years using substances and connectivity with NAcc are nodes in the right frontoparietal network, which is central to cognitive control. The coupling of reward and cognitive control networks may be a mechanism through which earlier onset of substance use is related to brain function over time, a trajectory that may be implicated in subsequent substance use disorders. Copyright © 2015 The Authors. Published by Elsevier Ltd.. All rights reserved.
-
Rapuano, Kristina M; Huckins, Jeremy F; Sargent, James D; Heatherton, Todd F; Kelley, William M
2016-06-01
The prevalence of adolescent obesity has increased dramatically over the past three decades, and research has documented that the number of television shows viewed during childhood is associated with greater risk for obesity. In particular, considerable evidence suggests that exposure to food marketing promotes eating habits that contribute to obesity. The present study examines neural responses to dynamic food commercials in overweight and healthy-weight adolescents using functional magnetic resonance imaging (fMRI). Compared with non-food commercials, food commercials more strongly engaged regions involved in attention and saliency detection (occipital lobe, precuneus, superior temporal gyri, and right insula) and in processing rewards [left and right nucleus accumbens (NAcc) and left orbitofrontal cortex (OFC)]. Activity in the left OFC and right insula further correlated with subjects' percent body fat at the time of the scan. Interestingly, this reward-related activity to food commercials was accompanied by the additional recruitment of mouth-specific somatosensory-motor cortices-a finding that suggests the intriguing possibility that higher-adiposity adolescents mentally simulate eating behaviors and offers a potential neural mechanism for the formation and reinforcement of unhealthy eating habits that may hamper an individual's ability lose weight later in life. © The Author 2015. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.
-
Self-stimulation in the rat: quantitative characteristics of the reward pathway.
Gallistel, C R
1978-12-01
Quantitative characteristics of the neural pathway that carries the reinforcing signal in electrical self-stimulation of the brain were established by finding which combinations of stimulation parameters give the same performance in a runway. The reward for each run was a train of evenly spaced monophasic cathodal pulses from a monopolar electrode. With train duration and pulse frequency held constant, the required current was a hyperbolic function of pulse duration, with chronaxie c approximately 1.5 msec. With pulse duration held constant, the required strength of the train (the charge delivered per second) was a hyperbolic function of train duration, with chronaxie C approximately 500 msec. To a first approximation, the values of c and C were independent of the choice either of train duration and pulse frequency or of pulse duration, respectively. Hence, the current intensity required by any choice of train duration, pulse frequency, and pulse duration dependent on only two basic parameters, c and C, and one quantity, Qi, the required impulse charge. These may reflect, respectively, current integration by directly excited neurons; temporal integration of neural activity by synaptic processes in a neural network; and the peak of the impulse response of the network, assuming that the network has linear dynamics and that the reward depends on the peak of the output of the network.
-
The relationship between reward and punishment processing and the 5-HT1A receptor as shown by PET.
Faulkner, Paul; Selvaraj, Sudhakar; Pine, Alex; Howes, Oliver D; Roiser, Jonathan P
2014-07-01
The serotonin (5-HT) system has been reported to be involved in decision-making. A key component of this neurotransmitter system is the 5-HT1A receptor, and research is beginning to show how this receptor can influence decision-making. However, this relationship has rarely been studied in humans. This study assessed whether individual variability in 5-HT1A availability correlates with decision-making in healthy volunteers. We measured regional availability of the 5-HT1A receptor in the hippocampal complex and striatum using positron emission tomography and correlated this with performance on two decision-making tasks measuring sensitivity to probability, rewards and punishments and temporal discounting, respectively. No relationship between decision-making behaviour and 5-HT1A availability in the striatum was found. However, a positive correlation was detected between participants' 5-HT1A availability in the hippocampal complex and their sensitivity to the probability of winning. Furthermore, there was a negative correlation between the degree to which participants discounted future rewards and 5-HT1A availability in the hippocampal complex. These data support a role for the 5-HT1A receptor in the aberrant decision-making that can occur in neuropsychiatric disorders such as depression.
-
Tamosiunaite, Minija; Asfour, Tamim; Wörgötter, Florentin
2009-03-01
Reinforcement learning methods can be used in robotics applications especially for specific target-oriented problems, for example the reward-based recalibration of goal directed actions. To this end still relatively large and continuous state-action spaces need to be efficiently handled. The goal of this paper is, thus, to develop a novel, rather simple method which uses reinforcement learning with function approximation in conjunction with different reward-strategies for solving such problems. For the testing of our method, we use a four degree-of-freedom reaching problem in 3D-space simulated by a two-joint robot arm system with two DOF each. Function approximation is based on 4D, overlapping kernels (receptive fields) and the state-action space contains about 10,000 of these. Different types of reward structures are being compared, for example, reward-on- touching-only against reward-on-approach. Furthermore, forbidden joint configurations are punished. A continuous action space is used. In spite of a rather large number of states and the continuous action space these reward/punishment strategies allow the system to find a good solution usually within about 20 trials. The efficiency of our method demonstrated in this test scenario suggests that it might be possible to use it on a real robot for problems where mixed rewards can be defined in situations where other types of learning might be difficult.
-
Transcriptomic analysis of instinctive and learned reward-related behaviors in honey bees
Naeger, Nicholas L.
2016-01-01
ABSTRACT We used transcriptomics to compare instinctive and learned, reward-based honey bee behaviors with similar spatio-temporal components: mating flights by males (drones) and time-trained foraging flights by females (workers), respectively. Genome-wide gene expression profiling via RNA sequencing was performed on the mushroom bodies, a region of the brain known for multi-modal sensory integration and responsive to various types of reward. Differentially expressed genes (DEGs) associated with the onset of mating (623 genes) were enriched for the gene ontology (GO) categories of Transcription, Unfolded Protein Binding, Post-embryonic Development, and Neuron Differentiation. DEGs associated with the onset of foraging (473) were enriched for Lipid Transport, Regulation of Programmed Cell Death, and Actin Cytoskeleton Organization. These results demonstrate that there are fundamental molecular differences between similar instinctive and learned behaviors. In addition, there were 166 genes with strong similarities in expression across the two behaviors – a statistically significant overlap in gene expression, also seen in Weighted Gene Co-Expression Network Analysis. This finding indicates that similar instinctive and learned behaviors also share common molecular architecture. This common set of DEGs was enriched for Regulation of RNA Metabolic Process, Transcription Factor Activity, and Response to Ecdysone. These findings provide a starting point for better understanding the relationship between instincts and learned behaviors. In addition, because bees collect food for their colony rather than for themselves, these results also support the idea that altruistic behavior relies, in part, on elements of brain reward systems associated with selfish behavior. PMID:27852762
-
Taylor, Johanna; Aldridge, Jan
2017-12-16
Children's hospices are a key provider of palliative care for children and young people with life-limiting and life-threatening conditions. However, despite recent policy attention to the provision of paediatric palliative care, little is known about the role of children's hospice staff and the factors that may impact on their wellbeing at work. This study explored the rewards and challenges of working in a children's hospice with an aim to identify staff support and development needs. We conducted an exploratory, qualitative study involving thematic analysis of semi-structured interviews with 34 staff and three focus groups with 17 staff working in a multi-disciplinary care team in a UK children's hospice. Participants identified rewards and challenges related to the direct work of caring for children and their families; team dynamics and organisational structures; and individual resilience and job motivation. Participants described the work as emotionally intensive and multi-faceted; 'getting it right' for children was identified as a strong motivator and reward, but also a potential stressor as staff strived to maintain high standards of personalised and emotional care. Other factors were identified as both a reward and stressor, including team functioning, the allocation of work, meeting parent expectations, and the hospice environment. Many participants identified training needs for different aspects of the role to help them feel more confident and competent. Participants also expressed concerns about work-related stress, both for themselves and for colleagues, but felt unable to discuss this at work. Informal support from colleagues and group clinical reflection were identified as primary resources to reflect on and learn from work and for emotional support. However, opportunities for this were limited. Providing regular, structured, and dedicated clinical reflection provides a mechanism through which children's hospice staff can come together for support and learning, and demonstrates an organisational commitment to staff wellbeing and development. Being aware of children's hospice specific rewards and challenges can help to ensure that staff feel supported and competent in their role. Breaking down barriers to discussing work-related stress and enhancing awareness about early signs of burnout is also important.
-
Role of habenula and amygdala dysfunction in Parkinson disease patients with punding.
Markovic, Vladana; Agosta, Federica; Canu, Elisa; Inuggi, Alberto; Petrovic, Igor; Stankovic, Iva; Imperiale, Francesca; Stojkovic, Tanja; Kostic, Vladimir S; Filippi, Massimo
2017-06-06
To assess whether a functional dysregulation of the habenula and amygdala, as modulators of the reward brain circuit, contributes to Parkinson disease (PD) punding. Structural and resting-state functional MRI were obtained from 22 patients with PD punding, 30 patients with PD without any impulsive-compulsive behavior (ICB) matched for disease stage and duration, motor impairment, and cognitive status, and 30 healthy controls. Resting-state functional connectivity of the habenula and amygdala bilaterally was assessed using a seed-based approach. Habenula and amygdala volumes and cortical thickness measures were obtained. Compared to both healthy controls and PD cases without any ICB (PD-no ICB), PD-punding patients showed higher functional connectivity of habenula and amygdala with thalamus and striatum bilaterally, and lower connectivity between bilateral habenula and left frontal and precentral cortices. In PD-punding relative to PD-no ICB patients, a lower functional connectivity between right amygdala and hippocampus was also observed. Habenula and amygdala volumes were not different among groups. PD-punding patients showed a cortical thinning of the left superior frontal and precentral gyri and right middle temporal gyrus and isthmus cingulate compared to healthy controls, and of the right inferior frontal gyrus compared to both controls and PD-no ICB patients. A breakdown of the connectivity among the crucial nodes of the reward circuit (i.e., habenula, amygdala, basal ganglia, frontal cortex) might be a contributory factor to punding in PD. This study provides potential instruments to detect and monitor punding in patients with PD. © 2017 American Academy of Neurology.
-
How musical training affects cognitive development: rhythm, reward and other modulating variables.
Miendlarzewska, Ewa A; Trost, Wiebke J
2013-01-01
Musical training has recently gained additional interest in education as increasing neuroscientific research demonstrates its positive effects on brain development. Neuroimaging revealed plastic changes in the brains of adult musicians but it is still unclear to what extent they are the product of intensive music training rather than of other factors, such as preexisting biological markers of musicality. In this review, we synthesize a large body of studies demonstrating that benefits of musical training extend beyond the skills it directly aims to train and last well into adulthood. For example, children who undergo musical training have better verbal memory, second language pronunciation accuracy, reading ability and executive functions. Learning to play an instrument as a child may even predict academic performance and IQ in young adulthood. The degree of observed structural and functional adaptation in the brain correlates with intensity and duration of practice. Importantly, the effects on cognitive development depend on the timing of musical initiation due to sensitive periods during development, as well as on several other modulating variables. Notably, we point to motivation, reward and social context of musical education, which are important yet neglected factors affecting the long-term benefits of musical training. Further, we introduce the notion of rhythmic entrainment and suggest that it may represent a mechanism supporting learning and development of executive functions. It also hones temporal processing and orienting of attention in time that may underlie enhancements observed in reading and verbal memory. We conclude that musical training uniquely engenders near and far transfer effects, preparing a foundation for a range of skills, and thus fostering cognitive development.
-
Slater, Jessica; Ashley, Richard; Tierney, Adam; Kraus, Nina
2018-01-01
Musical rhythm engages motor and reward circuitry that is important for cognitive control, and there is evidence for enhanced inhibitory control in musicians. We recently revealed an inhibitory control advantage in percussionists compared with vocalists, highlighting the potential importance of rhythmic expertise in mediating this advantage. Previous research has shown that better inhibitory control is associated with less variable performance in simple sensorimotor synchronization tasks; however, this relationship has not been examined through the lens of rhythmic expertise. We hypothesize that the development of rhythm skills strengthens inhibitory control in two ways: by fine-tuning motor networks through the precise coordination of movements "in time" and by activating reward-based mechanisms, such as predictive processing and conflict monitoring, which are involved in tracking temporal structure in music. Here, we assess adult percussionists and nonpercussionists on inhibitory control, selective attention, basic drumming skills (self-paced, paced, and continuation drumming), and cortical evoked responses to an auditory stimulus presented on versus off the beat of music. Consistent with our hypotheses, we find that better inhibitory control is correlated with more consistent drumming and enhanced neural tracking of the musical beat. Drumming variability and the neural index of beat alignment each contribute unique predictive power to a regression model, explaining 57% of variance in inhibitory control. These outcomes present the first evidence that enhanced inhibitory control in musicians may be mediated by rhythmic expertise and provide a foundation for future research investigating the potential for rhythm-based training to strengthen cognitive function.
-
Time to rethink the neural mechanisms of learning and memory.
Gallistel, Charles R; Balsam, Peter D
2014-02-01
Most studies in the neurobiology of learning assume that the underlying learning process is a pairing - dependent change in synaptic strength that requires repeated experience of events presented in close temporal contiguity. However, much learning is rapid and does not depend on temporal contiguity, which has never been precisely defined. These points are well illustrated by studies showing that the temporal relations between events are rapidly learned- even over long delays- and that this knowledge governs the form and timing of behavior. The speed with which anticipatory responses emerge in conditioning paradigms is determined by the information that cues provide about the timing of rewards. The challenge for understanding the neurobiology of learning is to understand the mechanisms in the nervous system that encode information from even a single experience, the nature of the memory mechanisms that can encode quantities such as time, and how the brain can flexibly perform computations based on this information. Copyright © 2013 Elsevier Inc. All rights reserved.
-
Voon, Valerie; Morris, Laurel S; Irvine, Michael A; Ruck, Christian; Worbe, Yulia; Derbyshire, Katherine; Rankov, Vladan; Schreiber, Liana Rn; Odlaug, Brian L; Harrison, Neil A; Wood, Jonathan; Robbins, Trevor W; Bullmore, Edward T; Grant, Jon E
2015-03-01
Pathological behaviors toward drugs and food rewards have underlying commonalities. Risk-taking has a fourfold pattern varying as a function of probability and valence leading to the nonlinearity of probability weighting with overweighting of small probabilities and underweighting of large probabilities. Here we assess these influences on risk-taking in patients with pathological behaviors toward drug and food rewards and examine structural neural correlates of nonlinearity of probability weighting in healthy volunteers. In the anticipation of rewards, subjects with binge eating disorder show greater risk-taking, similar to substance-use disorders. Methamphetamine-dependent subjects had greater nonlinearity of probability weighting along with impaired subjective discrimination of probability and reward magnitude. Ex-smokers also had lower risk-taking to rewards compared with non-smokers. In the anticipation of losses, obesity without binge eating had a similar pattern to other substance-use disorders. Obese subjects with binge eating also have impaired discrimination of subjective value similar to that of the methamphetamine-dependent subjects. Nonlinearity of probability weighting was associated with lower gray matter volume in dorsolateral and ventromedial prefrontal cortex and orbitofrontal cortex in healthy volunteers. Our findings support a distinct subtype of binge eating disorder in obesity with similarities in risk-taking in the reward domain to substance use disorders. The results dovetail with the current approach of defining mechanistically based dimensional approaches rather than categorical approaches to psychiatric disorders. The relationship to risk probability and valence may underlie the propensity toward pathological behaviors toward different types of rewards.
-
Voon, Valerie; Morris, Laurel S; Irvine, Michael A; Ruck, Christian; Worbe, Yulia; Derbyshire, Katherine; Rankov, Vladan; Schreiber, Liana RN; Odlaug, Brian L; Harrison, Neil A; Wood, Jonathan; Robbins, Trevor W; Bullmore, Edward T; Grant, Jon E
2015-01-01
Pathological behaviors toward drugs and food rewards have underlying commonalities. Risk-taking has a fourfold pattern varying as a function of probability and valence leading to the nonlinearity of probability weighting with overweighting of small probabilities and underweighting of large probabilities. Here we assess these influences on risk-taking in patients with pathological behaviors toward drug and food rewards and examine structural neural correlates of nonlinearity of probability weighting in healthy volunteers. In the anticipation of rewards, subjects with binge eating disorder show greater risk-taking, similar to substance-use disorders. Methamphetamine-dependent subjects had greater nonlinearity of probability weighting along with impaired subjective discrimination of probability and reward magnitude. Ex-smokers also had lower risk-taking to rewards compared with non-smokers. In the anticipation of losses, obesity without binge eating had a similar pattern to other substance-use disorders. Obese subjects with binge eating also have impaired discrimination of subjective value similar to that of the methamphetamine-dependent subjects. Nonlinearity of probability weighting was associated with lower gray matter volume in dorsolateral and ventromedial prefrontal cortex and orbitofrontal cortex in healthy volunteers. Our findings support a distinct subtype of binge eating disorder in obesity with similarities in risk-taking in the reward domain to substance use disorders. The results dovetail with the current approach of defining mechanistically based dimensional approaches rather than categorical approaches to psychiatric disorders. The relationship to risk probability and valence may underlie the propensity toward pathological behaviors toward different types of rewards. PMID:25270821
-
Valuation of opportunity costs by rats working for rewarding electrical brain stimulation.
Solomon, Rebecca Brana; Conover, Kent; Shizgal, Peter
2017-01-01
Pursuit of one goal typically precludes simultaneous pursuit of another. Thus, each exclusive activity entails an "opportunity cost:" the forgone benefits from the next-best activity eschewed. The present experiment estimates, in laboratory rats, the function that maps objective opportunity costs into subjective ones. In an operant chamber, rewarding electrical brain stimulation was delivered when the cumulative time a lever had been depressed reached a criterion duration. The value of the activities forgone during this duration is the opportunity cost of the electrical reward. We determined which of four functions best describes how objective opportunity costs, expressed as the required duration of lever depression, are translated into their subjective equivalents. The simplest account is the identity function, which equates subjective and objective opportunity costs. A variant of this function called the "sigmoidal-slope function," converges on the identity function at longer durations but deviates from it at shorter durations. The sigmoidal-slope function has the form of a hockey stick. The flat "blade" denotes a range over which opportunity costs are subjectively equivalent; these durations are too short to allow substitution of more beneficial activities. The blade extends into an upward-curving portion over which costs become discriminable and finally into the straight "handle," over which objective and subjective costs match. The two remaining functions are based on hyperbolic and exponential temporal discounting, respectively. The results are best described by the sigmoidal-slope function. That this is so suggests that different principles of intertemporal choice are involved in the evaluation of time spent working for a reward or waiting for its delivery. The subjective opportunity-cost function plays a key role in the evaluation and selection of goals. An accurate description of its form and parameters is essential to successful modeling and prediction of instrumental performance and reward-related decision making.
-
Schmidt, Burkhard; Bosch, Jos A; Jarczok, Marc N; Herr, Raphael M; Loerbroks, Adrian; van Vianen, Annelies E M; Fischer, Joachim E
2015-01-15
Job stress is a predictor of cardiovascular disease incidence and mortality and the metabolic syndrome (MetS) represents one of the key pathways potentially underlying those associations. Effort-reward imbalance (ERI) represents one of the most influential theoretical work stress models, but evidence on its relationship with MetS remains sparse and with only limited generalizability. We therefore aimed to determine this association in a large occupational sample with different occupational groups. The present study used cross-sectional data from an industrial sample in Germany (n=4141). ERI was assessed by a validated 10-item questionnaire. MetS was defined according to a joined interim statement of six expert associations involved with MetS, stating that three out of five risk factors (raised blood pressure, elevated triglycerides, low high density lipoprotein, raised fasting glucose and central obesity) qualify a patient for MetS. Multivariable associations of ERI, and its subcomponents "effort" and "reward", with MetS were estimated by logistic regression-based multivariate odds ratios (ORs) with 95% confidence intervals (CIs). ERI (continuous z-score) was positively associated with MetS (zERI: OR=1.14, 95% CI=1.03-1.26). The association was more prominent in males (zERI: OR 1.20, 95% CI=1.07-1.33) and in younger employees (age 18-49 zERI: OR=1.24, 95% CI=1.09-1.40). Analysis of the ERI subcomponents yielded weak associations of both effort (zEffort: OR=1.12, 95% CI=1.00-1.25) and reward (zReward: OR=0.92, 95% CI=0.84-1.00) with MetS. ERI is associated with increased occurrence of MetS, in particular among younger men. Further longitudinal studies are needed to determine the temporal relation of these associations. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.
-
Reward and aversion in a heterogeneous midbrain dopamine system.
Lammel, Stephan; Lim, Byung Kook; Malenka, Robert C
2014-01-01
The ventral tegmental area (VTA) is a heterogeneous brain structure that serves a central role in motivation and reward processing. Abnormalities in the function of VTA dopamine (DA) neurons and the targets they influence are implicated in several prominent neuropsychiatric disorders including addiction and depression. Recent studies suggest that the midbrain DA system is composed of anatomically and functionally heterogeneous DA subpopulations with different axonal projections. These findings may explain a number of previously confusing observations that suggested a role for DA in processing both rewarding as well as aversive events. Here we will focus on recent advances in understanding the neural circuits mediating reward and aversion in the VTA and how stress as well as drugs of abuse, in particular cocaine, alter circuit function within a heterogeneous midbrain DA system. This article is part of a Special Issue entitled 'NIDA 40th Anniversary Issue'. Copyright © 2013 Elsevier Ltd. All rights reserved.
-
The role of high-frequency oscillatory activity in reward processing and learning.
Marco-Pallarés, Josep; Münte, Thomas F; Rodríguez-Fornells, Antoni
2015-02-01
Oscillatory activity has been proposed as a key mechanism in the integration of brain activity of distant structures. Particularly, high frequency brain oscillatory activity in the beta and gamma range has received increasing interest in the domains of attention and memory. In addition, a number of recent studies have revealed an increase of beta-gamma activity (20-35 Hz) after unexpected or relevant positive reward outcomes. In the present manuscript we review the literature on this phenomenon and we propose that this activity is a brain signature elicited by unexpected positive outcomes in order to transmit a fast motivational value signal to the reward network. In addition, we hypothesize that beta-gamma oscillatory activity indexes the interaction between attentional and emotional systems, and that it directly reflects the appearance of unexpected positive rewards in learning-related contexts. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
Hedging Your Bets by Learning Reward Correlations in the Human Brain
Wunderlich, Klaus; Symmonds, Mkael; Bossaerts, Peter; Dolan, Raymond J.
2011-01-01
Summary Human subjects are proficient at tracking the mean and variance of rewards and updating these via prediction errors. Here, we addressed whether humans can also learn about higher-order relationships between distinct environmental outcomes, a defining ecological feature of contexts where multiple sources of rewards are available. By manipulating the degree to which distinct outcomes are correlated, we show that subjects implemented an explicit model-based strategy to learn the associated outcome correlations and were adept in using that information to dynamically adjust their choices in a task that required a minimization of outcome variance. Importantly, the experimentally generated outcome correlations were explicitly represented neuronally in right midinsula with a learning prediction error signal expressed in rostral anterior cingulate cortex. Thus, our data show that the human brain represents higher-order correlation structures between rewards, a core adaptive ability whose immediate benefit is optimized sampling. PMID:21943609
-
Sleep and Circadian Contributions to Adolescent Alcohol Use Disorder
Hasler, Brant P.; Soehner, Adriane M.; Clark, Duncan B.
2014-01-01
Adolescence is a time of marked changes across sleep, circadian rhythms, brain function, and alcohol use. Starting at puberty, adolescents’ endogenous circadian rhythms and preferred sleep times shift later, often leading to a mismatch with the schedules imposed by secondary education. This mismatch induces circadian misalignment and sleep loss, which have been associated with affect dysregulation, increased drug and alcohol use, and other risk-taking behaviors in adolescents and adults. In parallel to developmental changes in sleep, adolescent brains are undergoing structural and functional changes in the circuits subserving the pursuit and processing of rewards. These developmental changes in reward processing likely contribute to the initiation of alcohol use during adolescence. Abundant evidence indicates that sleep and circadian rhythms modulate reward function, suggesting that adolescent sleep and circadian disturbance may contribute to altered reward function, and in turn, alcohol involvement. In this review, we summarize the relevant evidence and propose that these parallel developmental changes in sleep, circadian rhythms, and neural processing of reward interact to increase risk for alcohol use disorder (AUD). PMID:25442171
-
Monteleone, Alessio Maria; Castellini, Giovanni; Volpe, Umberto; Ricca, Valdo; Lelli, Lorenzo; Monteleone, Palmiero; Maj, Mario
2018-01-03
Anorexia nervosa and bulimia nervosa are severe eating disorders whose etiopathogenesis is still unknown. Clinical features suggest that eating disorders may develop as reward-dependent syndromes, since eating less food is perceived as rewarding in anorexia nervosa while consumption of large amounts of food during binge episodes in bulimia nervosa aims at reducing the patient's negative emotional states. Therefore, brain reward mechanisms have been a major focus of research in the attempt to contribute to the comprehension of the pathophysiology of these disorders. Structural brain imaging data provided the evidence that brain reward circuits may be altered in patients with anorexia or bulimia nervosa. Similarly, functional brain imaging studies exploring the activation of brain reward circuits by food stimuli as well as by stimuli recognized to be potentially rewarding for eating disordered patients, such as body image cues or stimuli related to food deprivation and physical hyperactivity, showed several dysfunctions in ED patients. Moreover, very recently, it has been demonstrated that some of the biochemical homeostatic modulators of eating behavior are also implicated in the regulation of food-related and non-food-related reward, representing a possible link between the aberrant behaviors of ED subjects and their hypothesized deranged reward processes. In particular, changes in leptin and ghrelin occur in patients with anorexia or bulimia nervosa and have been suggested to represent not only homeostatic adaptations to an altered energy balance but to contribute also to the acquisition and/or maintenance of persistent starvation, binge eating and physical hyperactivity, which are potentially rewarding for ED patients. On the basis of such findings new pathogenetic models of EDs have been proposed, and these models may provide new theoretical basis for the development of innovative treatment strategies, either psychological and pharmacological, with the aim to improve the outcomes of so severe disabling disorders. Copyright © 2017 Elsevier Inc. All rights reserved.
-
Spetter, Maartje S; Feld, Gordon B; Thienel, Matthias; Preissl, Hubert; Hege, Maike A; Hallschmid, Manfred
2018-02-09
The hypothalamic neurohormone oxytocin decreases food intake via largely unexplored mechanisms. We investigated the central nervous mediation of oxytocin's hypophagic effect in comparison to its impact on the processing of generalized rewards. Fifteen fasted normal-weight, young men received intranasal oxytocin (24 IU) or placebo before functional magnetic resonance imaging (fMRI) measurements of brain activity during exposure to food stimuli and a monetary incentive delay task (MID). Subsequently, ad-libitum breakfast intake was assessed. Oxytocin compared to placebo increased activity in the ventromedial prefrontal cortex, supplementary motor area, anterior cingulate, and ventrolateral prefrontal cortices in response to high- vs. low-calorie food images in the fasted state, and reduced calorie intake by 12%. During anticipation of monetary rewards, oxytocin compared to placebo augmented striatal, orbitofrontal and insular activity without altering MID performance. We conclude that during the anticipation of generalized rewards, oxytocin stimulates dopaminergic reward-processing circuits. In contrast, oxytocin restrains food intake by enhancing the activity of brain regions that exert cognitive control, while concomitantly increasing the activity of structures that process food reward value. This pattern points towards a specific role of oxytocin in the regulation of eating behaviour in humans that might be of relevance for potential clinical applications.
-
Jiménez-Murcia, Susana; Fernández-Aranda, Fernando; Mestre-Bach, Gemma; Granero, Roser; Tárrega, Salomé; Torrubia, Rafael; Aymamí, Neus; Gómez-Peña, Mónica; Soriano-Mas, Carles; Steward, Trevor; Moragas, Laura; Baño, Marta; Del Pino-Gutiérrez, Amparo; Menchón, José M
2017-06-01
Most individuals will gamble during their lifetime, yet only a select few will develop gambling disorder. Gray's Reinforcement Sensitivity Theory holds promise for providing insight into gambling disorder etiology and symptomatology as it ascertains that neurobiological differences in reward and punishment sensitivity play a crucial role in determining an individual's affect and motives. The aim of the study was to assess a mediational pathway, which included patients' sex, personality traits, reward and punishment sensitivity, and gambling-severity variables. The Sensitivity to Punishment and Sensitivity to Reward Questionnaire, the South Oaks Gambling Screen, the Symptom Checklist-Revised, and the Temperament and Character Inventory-Revised were administered to a sample of gambling disorder outpatients (N = 831), diagnosed according to DSM-5 criteria, attending a specialized outpatient unit. Sociodemographic variables were also recorded. A structural equation model found that both reward and punishment sensitivity were positively and directly associated with increased gambling severity, sociodemographic variables, and certain personality traits while also revealing a complex mediational role for these dimensions. To this end, our findings suggest that the Sensitivity to Punishment and Sensitivity to Reward Questionnaire could be a useful tool for gaining a better understanding of different gambling disorder phenotypes and developing tailored interventions.
-
Review. Neurobiological mechanisms for opponent motivational processes in addiction.
Koob, George F; Le Moal, Michel
2008-10-12
The conceptualization of drug addiction as a compulsive disorder with excessive drug intake and loss of control over intake requires motivational mechanisms. Opponent process as a motivational theory for the negative reinforcement of drug dependence has long required a neurobiological explanation. Key neurochemical elements involved in reward and stress within basal forebrain structures involving the ventral striatum and extended amygdala are hypothesized to be dysregulated in addiction to convey the opponent motivational processes that drive dependence. Specific neurochemical elements in these structures include not only decreases in reward neurotransmission such as dopamine and opioid peptides in the ventral striatum, but also recruitment of brain stress systems such as corticotropin-releasing factor (CRF), noradrenaline and dynorphin in the extended amygdala. Acute withdrawal from all major drugs of abuse produces increases in reward thresholds, anxiety-like responses and extracellular levels of CRF in the central nucleus of the amygdala. CRF receptor antagonists block excessive drug intake produced by dependence. A brain stress response system is hypothesized to be activated by acute excessive drug intake, to be sensitized during repeated withdrawal, to persist into protracted abstinence and to contribute to stress-induced relapse. The combination of loss of reward function and recruitment of brain stress systems provides a powerful neurochemical basis for the long hypothesized opponent motivational processes responsible for the negative reinforcement driving addiction.
-
The role of dopamine in risk taking: a specific look at Parkinson’s disease and gambling
Clark, Crystal A.; Dagher, Alain
2014-01-01
An influential model suggests that dopamine signals the difference between predicted and experienced reward. In this way, dopamine can act as a learning signal that can shape behaviors to maximize rewards and avoid punishments. Dopamine is also thought to invigorate reward seeking behavior. Loss of dopamine signaling is the major abnormality in Parkinson’s disease. Dopamine agonists have been implicated in the occurrence of impulse control disorders in Parkinson’s disease patients, the most common being pathological gambling, compulsive sexual behavior, and compulsive buying. Recently, a number of functional imaging studies investigating impulse control disorders in Parkinson’s disease have been published. Here we review this literature, and attempt to place it within a decision-making framework in which potential gains and losses are evaluated to arrive at optimum choices. We also provide a hypothetical but still incomplete model on the effect of dopamine agonist treatment on these value and risk assessments. Two of the main brain structures thought to be involved in computing aspects of reward and loss are the ventral striatum (VStr) and the insula, both dopamine projection sites. Both structures are consistently implicated in functional brain imaging studies of pathological gambling in Parkinson’s disease. PMID:24910600
-
Intrinsic rewards experienced by a group of dentists working with underserved populations.
Gardner, S P; Roberts-Thomson, K F; Winning, T A; Peterson, R
2014-09-01
The aim of this study was to explore, using qualitative methods, the intrinsic reasons why dentists work with underserved groups. Minority and marginalized groups of Australians suffer a greater burden of dental disease than the general population due to disparities in accessing care. Recruitment and retention of dentists to care for underserved groups is problematic due to personal, professional and structural reasons. What drives dentists to work with underserved groups is not widely known. Sixteen dentists were recruited using 'snowball' purposeful sampling. Semi-structured in-depth interviews were conducted. Thematic analysis was conducted on the transcriptions to identify themes. Five key themes emerged: (1) 'tapped on the shoulder', being personally approached or invited; (2) 'dental school experience', the challenges faced as a student; (3) 'empathic concern', the non-judgemental expressions of care toward others; (4) 'resilience', the ability to bounce back after setbacks; (5) 'intrinsic reward', the personal gain and satisfaction received. This study focuses on the intrinsic rewards which were found to be simple, unexpected, and associated with relieving pain, community engagement and making a difference. Emphasizing personal fulfilment and intrinsic reward could be useful when promoting dentistry as a career and when encouraging graduates to consider working with disadvantaged groups. © 2014 Australian Dental Association.
-
Chronic Motivational State Interacts with Task Reward Structure in Dynamic Decision-Making
Cooper, Jessica A.; Worthy, Darrell A.; Maddox, W. Todd
2015-01-01
Research distinguishes between a habitual, model-free system motivated toward immediately rewarding actions, and a goal-directed, model-based system motivated toward actions that improve future state. We examined the balance of processing in these two systems during state-based decision-making. We tested a regulatory fit hypothesis (Maddox & Markman, 2010) that predicts that global trait motivation affects the balance of habitual- vs. goal-directed processing but only through its interaction with the task framing as gain-maximization or loss-minimization. We found support for the hypothesis that a match between an individual’s chronic motivational state and the task framing enhances goal-directed processing, and thus state-based decision-making. Specifically, chronic promotion-focused individuals under gain-maximization and chronic prevention-focused individuals under loss-minimization both showed enhanced state-based decision-making. Computational modeling indicates that individuals in a match between global chronic motivational state and local task reward structure engaged more goal-directed processing, whereas those in a mismatch engaged more habitual processing. PMID:26520256
-
Narcissism is associated with weakened frontostriatal connectivity: a DTI study.
Chester, David S; Lynam, Donald R; Powell, David K; DeWall, C Nathan
2016-07-01
Narcissism is characterized by the search for affirmation and admiration from others. Might this motivation to find external sources of acclaim exist to compensate for neurostructural deficits that link the self with reward? Greater structural connectivity between brain areas that process self-relevant stimuli (i.e. the medial prefrontal cortex) and reward (i.e. the ventral striatum) is associated with fundamentally positive self-views. We predicted that narcissism would be associated with less integrity of this frontostriatal pathway. We used diffusion tensor imaging to assess the frontostriatal structural connectivity among 50 healthy undergraduates (32 females, 18 males) who also completed a measure of grandiose narcissism. White matter integrity in the frontostriatal pathway was negatively associated with narcissism. Our findings, while purely correlational, suggest that narcissism arises, in part, from a neural disconnect between the self and reward. The exhibitionism and immodesty of narcissists may then be a regulatory strategy to compensate for this neural deficit. © The Author (2015). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
Goh, Joshua O S; Su, Yu-Shiang; Tang, Yong-Jheng; McCarrey, Anna C; Tereshchenko, Alexander; Elkins, Wendy; Resnick, Susan M
2016-12-07
Aging compromises the frontal, striatal, and medial temporal areas of the reward system, impeding accurate value representation and feedback processing critical for decision making. However, substantial variability characterizes age-related effects on the brain so that some older individuals evince clear neurocognitive declines whereas others are spared. Moreover, the functional correlates of normative individual differences in older-adult value-based decision making remain unclear. We performed a functional magnetic resonance imaging study in 173 human older adults during a lottery choice task in which costly to more desirable stakes were depicted using low to high expected values (EVs) of points. Across trials that varied in EVs, participants decided to accept or decline the offered stakes to maximize total accumulated points. We found that greater age was associated with less optimal decisions, accepting stakes when losses were likely and declining stakes when gains were likely, and was associated with increased frontal activity for costlier stakes. Critically, risk preferences varied substantially across older adults and neural sensitivity to EVs in the frontal, striatal, and medial temporal areas dissociated risk-aversive from risk-taking individuals. Specifically, risk-averters increased neural responses to increasing EVs as stakes became more desirable, whereas risk-takers increased neural responses with decreasing EV as stakes became more costly. Risk preference also modulated striatal responses during feedback with risk-takers showing more positive responses to gains compared with risk-averters. Our findings highlight the frontal, striatal, and medial temporal areas as key neural loci in which individual differences differentially affect value-based decision-making ability in older adults. Frontal, striatal, and medial temporal functions implicated in value-based decision processing of rewards and costs undergo substantial age-related changes. However, age effects on brain function and cognition differ across individuals. How this normative variation relates to older-adult value-based decision making is unclear. We found that although the ability make optimal decisions declines with age, there is still much individual variability in how this deterioration occurs. Critically, whereas risk-averters showed increased neural activity to increasingly valuable stakes in frontal, striatal, and medial temporal areas, risk-takers instead increased activity as stakes became more costly. Such distinct functional decision-making processing in these brain regions across normative older adults may reflect individual differences in susceptibility to age-related brain changes associated with incipient cognitive impairment. Copyright © 2016 the authors 0270-6474/16/3612498-12$15.00/0.
-
Exploiting risk-reward structures in decision making under uncertainty.
Leuker, Christina; Pachur, Thorsten; Hertwig, Ralph; Pleskac, Timothy J
2018-06-01
People often have to make decisions under uncertainty-that is, in situations where the probabilities of obtaining a payoff are unknown or at least difficult to ascertain. One solution to this problem is to infer the probability from the magnitude of the potential payoff and thus exploit the inverse relationship between payoffs and probabilities that occurs in many domains in the environment. Here, we investigated how the mind may implement such a solution: (1) Do people learn about risk-reward relationships from the environment-and if so, how? (2) How do learned risk-reward relationships impact preferences in decision-making under uncertainty? Across three experiments (N = 352), we found that participants can learn risk-reward relationships from being exposed to choice environments with a negative, positive, or uncorrelated risk-reward relationship. They were able to learn the associations both from gambles with explicitly stated payoffs and probabilities (Experiments 1 & 2) and from gambles about epistemic events (Experiment 3). In subsequent decisions under uncertainty, participants often exploited the learned association by inferring probabilities from the magnitudes of the payoffs. This inference systematically influenced their preferences under uncertainty: Participants who had been exposed to a negative risk-reward relationship tended to prefer the uncertain option over a smaller sure option for low payoffs, but not for high payoffs. This pattern reversed in the positive condition and disappeared in the uncorrelated condition. This adaptive change in preferences is consistent with the use of the risk-reward heuristic. Copyright © 2018 Elsevier B.V. All rights reserved.
-
NASA Astrophysics Data System (ADS)
Len, Patrick M.
Electronic response systems ("clickers") are used in introductory astronomy classes as a real-time assessment tool. Different reward structures for student responses to clicker questions are used to motivate individual participation or group collaboration before responding. The impact of two reward structures on student behavior and learning is investigated. This study finds that a success-bonus incentive (in which individual participation points are doubled when the class attains a threshold success rate) strongly motivated students to collaborate, whereas a participation-only credit (no success-bonus) incentive resulted in one-third of the students answering individually without collaboration. With a participation-only incentive, students who answered individually ("self-testers") were found to have more positive attitudes toward astronomy and science, and higher self-confidence in their learning than students who interacted before answering without a success-bonus incentive ("collaborators"). These collaborators experienced downward shifts in attitudes and self-confidence, in contrast to the static attitudes and self-confidence of self-testers. The implication is that students with little or no background in science prefer to answer collaboratively rather than independently and that these students are also negatively impacted by a one-semester introductory astronomy course.
-
Wang, Lei; Zheng, Jiehui; Meng, Liang
2017-04-01
Although many studies have investigated the relationship between the amount of effort invested in a certain task and one's attitude towards the subsequent reward, whether exerted effort would impact one's expectation and evaluation of performance feedback itself still remains to be examined. In the present study, two types of calculation tasks that varied in the required effort were adopted, and we resorted to electroencephalography to probe the temporal dynamics of how exerted effort would affect one's anticipation and evaluation of performance feedback. In the high-effort condition, a more salient stimulus-preceding negativity was detected during the anticipation stage, which was accompanied with a more salient FRN/P300 complex (a more positive P300 and a less negative feedback-related negativity) in response to positive outcomes in the evaluation stage. These results suggested that when more effort was invested, an enhanced anticipatory attention would be paid toward one's task performance feedback and that positive outcomes would be subjectively valued to a greater extent.
-
Executive functioning and substance use in adolescence: Neurobiological and behavioral perspectives.
Kim-Spoon, Jungmeen; Kahn, Rachel E; Lauharatanahirun, Nina; Deater-Deckard, Kirby; Bickel, Warren K; Chiu, Pearl H; King-Casas, Brooks
2017-06-01
The current review is guided by the theoretical perspective that emphasizes the regulating role of executive functioning (Carver et al., 2009) and presents studies that elucidate the ways that executive functioning (inhibition and working memory) explain individual differences in adolescent substance use independently or by regulating the reactive system (reward and punishment sensitivity). Behavioral studies indicate that main effects of executive functioning on adolescent substance use are often nonsignificant or weak in effect sizes. In contrast, emerging evidence suggests consistent and stronger regulating effects of executive functioning over reward and punishment sensitivity. Functional neuroimaging studies reveal significant associations between executive functioning task-related hemodynamic responses and substance use with strong effect sizes. There is also direct evidence from studies testing statistical interactions of the regulating effects of EF-related brain activation, and indirect evidence in studies examining functional connectivity, temporal discounting, and reinforced control. We note key future directions and ways to address limitations in existing work. Copyright © 2017 Elsevier Ltd. All rights reserved.
-
Cooper, Nicole; Kable, Joseph W; Kim, B Kyu; Zauberman, Gal
2013-08-07
People vary widely in how much they discount delayed rewards, yet little is known about the sources of these differences. Here we demonstrate that neural activity in ventromedial prefrontal cortex (VMPFC) and ventral striatum (VS) when human subjects are asked to merely think about the future--specifically, to judge the subjective length of future time intervals--predicts delay discounting. High discounters showed lower activity for longer time delays, while low discounters showed the opposite pattern. Our results demonstrate that the correlation between VMPFC and VS activity and discounting occurs even in the absence of choices about future rewards, and does not depend on a person explicitly evaluating future outcomes or judging their self-relevance. This suggests a link between discounting and basic processes involved in thinking about the future, such as temporal perception. Our results also suggest that reducing impatience requires not suppression of VMPFC and VS activity altogether, but rather modulation of how these regions respond to the present versus the future.
-
A meta-analysis of temperament in eating disorders.
Atiye, Minna; Miettunen, Jouko; Raevuori-Helkamaa, Anu
2015-03-01
Although suggested as an important contributor to the development and maintenance of eating disorders, temperament has not previously been studied adopting a meta-analytical approach. We therefore pooled data (N = 14 studies; N = 3315 cases, N = 3395 controls) on Cloninger's temperament traits (novelty seeking, harm avoidance, reward dependence and persistence) in anorexia nervosa (AN), bulimia nervosa (BN), binge eating disorder (BED) and eating disorders not otherwise specified. Persistence was significantly higher than in the controls in all eating disorders except for BED the highest levels being observed in AN. Correspondingly, the highest effect sizes for harm avoidance were seen in AN. Novelty seeking was significantly elevated relative to the controls only in BN. Harm avoidance was significantly lower, and reward dependence was significantly higher in individuals who had recovered from AN than in those who remained ill. Future studies with a longitudinal design are needed to explore the temporal relationships between eating disorders and temperament traits. Copyright © 2014 John Wiley & Sons, Ltd and Eating Disorders Association.
-
Haenicke, Joachim; Yamagata, Nobuhiro; Zwaka, Hanna; Nawrot, Martin; Menzel, Randolf
2018-01-01
The mushroom body (MB) in insects is known as a major center for associative learning and memory, although exact locations for the correlating memory traces remain to be elucidated. Here, we asked whether presynaptic boutons of olfactory projection neurons (PNs) in the main input site of the MB undergo neuronal plasticity during classical odor-reward conditioning and correlate with the conditioned behavior. We simultaneously measured Ca 2+ responses in the boutons and conditioned behavioral responses to learned odors in honeybees. We found that the absolute amount of the neural change for the rewarded but not for the unrewarded odor was correlated with the behavioral learning rate across individuals. The temporal profile of the induced changes matched with odor response dynamics of the MB-associated inhibitory neurons, suggestive of activity modulation of boutons by this neural class. We hypothesize the circuit-specific neural plasticity relates to the learned value of the stimulus and underlies the conditioned behavior of the bees.
-
Cooper, Nicole; Kim, B. Kyu; Zauberman, Gal
2013-01-01
People vary widely in how much they discount delayed rewards, yet little is known about the sources of these differences. Here we demonstrate that neural activity in ventromedial prefrontal cortex (VMPFC) and ventral striatum (VS) when human subjects are asked to merely think about the future—specifically, to judge the subjective length of future time intervals—predicts delay discounting. High discounters showed lower activity for longer time delays, while low discounters showed the opposite pattern. Our results demonstrate that the correlation between VMPFC and VS activity and discounting occurs even in the absence of choices about future rewards, and does not depend on a person explicitly evaluating future outcomes or judging their self-relevance. This suggests a link between discounting and basic processes involved in thinking about the future, such as temporal perception. Our results also suggest that reducing impatience requires not suppression of VMPFC and VS activity altogether, but rather modulation of how these regions respond to the present versus the future. PMID:23926268
-
Riem, Madelon M E; van IJzendoorn, Marinus H; Tops, Mattie; Boksem, Maarten A S; Rombouts, Serge A R B; Bakermans-Kranenburg, Marian J
2012-04-01
Infant laughter is a rewarding experience. It activates neural reward circuits and promotes parental proximity and care, thus facilitating parent-infant attachment. The neuropeptide oxytocin might enhance the incentive salience of infant laughter by modulating neural circuits related to the perception of infant cues. In a randomized controlled trial with functional magnetic resonance imaging we investigated the influence of intranasally administered oxytocin on functional brain connectivity in response to infant laughter. Blood oxygenation level-dependent responses to infant laughter were measured in 22 nulliparous women who were administered oxytocin and 20 nulliparous women who were administered a placebo. Elevated oxytocin levels reduced activation in the amygdala during infant laughter and enhanced functional connectivity between the amygdala and the orbitofrontal cortex, the anterior cingulate, the hippocampus, the precuneus, the supramarginal gyri, and the middle temporal gyrus. Increased functional connectivity between the amygdala and regions involved in emotion regulation may reduce negative emotional arousal while enhancing the incentive salience of the infant laughter.
-
Riem, Madelon M E; van IJzendoorn, Marinus H; Tops, Mattie; Boksem, Maarten A S; Rombouts, Serge A R B; Bakermans-Kranenburg, Marian J
2012-01-01
Infant laughter is a rewarding experience. It activates neural reward circuits and promotes parental proximity and care, thus facilitating parent–infant attachment. The neuropeptide oxytocin might enhance the incentive salience of infant laughter by modulating neural circuits related to the perception of infant cues. In a randomized controlled trial with functional magnetic resonance imaging we investigated the influence of intranasally administered oxytocin on functional brain connectivity in response to infant laughter. Blood oxygenation level-dependent responses to infant laughter were measured in 22 nulliparous women who were administered oxytocin and 20 nulliparous women who were administered a placebo. Elevated oxytocin levels reduced activation in the amygdala during infant laughter and enhanced functional connectivity between the amygdala and the orbitofrontal cortex, the anterior cingulate, the hippocampus, the precuneus, the supramarginal gyri, and the middle temporal gyrus. Increased functional connectivity between the amygdala and regions involved in emotion regulation may reduce negative emotional arousal while enhancing the incentive salience of the infant laughter. PMID:22189289
-
The experience of art: insights from neuroimaging.
Nadal, Marcos
2013-01-01
The experience of art is a complex one. It emerges from the interaction of multiple cognitive and affective processes. Neuropsychological and neuroimaging studies are revealing the broadly distributed network of brain regions upon which it relies. This network can be divided into three functional components: (i) prefrontal, parietal, and temporal cortical regions support evaluative judgment, attentional processing, and memory retrieval; (ii) the reward circuit, including cortical, subcortical regions, and some of its regulators, is involved in the generation of pleasurable feelings and emotions, and the valuation and anticipation of reward; and (iii) attentional modulation of activity in low-, mid-, and high-level cortical sensory regions enhances the perceptual processing of certain features, relations, locations, or objects. Understanding how these regions act in concert to produce unique and moving art experiences and determining the impact of personal and cultural meaning and context on this network the biological foundation of the experience of art--remain future challenges. © 2013 Elsevier B.V. All rights reserved.
-
Relation of obesity to neural activation in response to food commercials.
Gearhardt, Ashley N; Yokum, Sonja; Stice, Eric; Harris, Jennifer L; Brownell, Kelly D
2014-07-01
Adolescents view thousands of food commercials annually, but the neural response to food advertising and its association with obesity is largely unknown. This study is the first to examine how neural response to food commercials differs from other stimuli (e.g. non-food commercials and television show) and to explore how this response may differ by weight status. The blood oxygen level-dependent functional magnetic resonance imaging activation was measured in 30 adolescents ranging from lean to obese in response to food and non-food commercials imbedded in a television show. Adolescents exhibited greater activation in regions implicated in visual processing (e.g. occipital gyrus), attention (e.g. parietal lobes), cognition (e.g. temporal gyrus and posterior cerebellar lobe), movement (e.g. anterior cerebellar cortex), somatosensory response (e.g. postcentral gyrus) and reward [e.g. orbitofrontal cortex and anterior cingulate cortex (ACC)] during food commercials. Obese participants exhibited less activation during food relative to non-food commercials in neural regions implicated in visual processing (e.g. cuneus), attention (e.g. posterior cerebellar lobe), reward (e.g. ventromedial prefrontal cortex and ACC) and salience detection (e.g. precuneus). Obese participants did exhibit greater activation in a region implicated in semantic control (e.g. medial temporal gyrus). These findings may inform current policy debates regarding the impact of food advertising to minors. © The Author (2013). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
Chantiluke, Kaylita; Christakou, Anastasia; Murphy, Clodagh M; Giampietro, Vincent; Daly, Eileen M; Ecker, Christina; Brammer, Michael; Murphy, Declan G; Rubia, Katya
2014-08-30
Attention Deficit Hyperactivity Disorder (ADHD) and Autism Spectrum Disorder (ASD) are often comorbid and share cognitive abnormalities in temporal foresight. A key question is whether shared cognitive phenotypes are based on common or different underlying pathophysiologies and whether comorbid patients have additive neurofunctional deficits, resemble one of the disorders or have a different pathophysiology. We compared age- and IQ-matched boys with non-comorbid ADHD (18), non-comorbid ASD (15), comorbid ADHD and ASD (13) and healthy controls (18) using functional magnetic resonance imaging (fMRI) during a temporal discounting task. Only the ASD and the comorbid groups discounted delayed rewards more steeply. The fMRI data showed both shared and disorder-specific abnormalities in the three groups relative to controls in their brain-behaviour associations. The comorbid group showed both unique and more severe brain-discounting associations than controls and the non-comorbid patient groups in temporal discounting areas of ventromedial and lateral prefrontal cortex, ventral striatum and anterior cingulate, suggesting that comorbidity is neither an endophenocopy of the two pure disorders nor an additive pathology. Copyright © 2014 The Authors. Published by Elsevier Ireland Ltd.. All rights reserved.
-
Zsuga, Judit; Biro, Klara; Papp, Csaba; Tajti, Gabor; Gesztelyi, Rudolf
2016-02-01
Reinforcement learning (RL) is a powerful concept underlying forms of associative learning governed by the use of a scalar reward signal, with learning taking place if expectations are violated. RL may be assessed using model-based and model-free approaches. Model-based reinforcement learning involves the amygdala, the hippocampus, and the orbitofrontal cortex (OFC). The model-free system involves the pedunculopontine-tegmental nucleus (PPTgN), the ventral tegmental area (VTA) and the ventral striatum (VS). Based on the functional connectivity of VS, model-free and model based RL systems center on the VS that by integrating model-free signals (received as reward prediction error) and model-based reward related input computes value. Using the concept of reinforcement learning agent we propose that the VS serves as the value function component of the RL agent. Regarding the model utilized for model-based computations we turned to the proactive brain concept, which offers an ubiquitous function for the default network based on its great functional overlap with contextual associative areas. Hence, by means of the default network the brain continuously organizes its environment into context frames enabling the formulation of analogy-based association that are turned into predictions of what to expect. The OFC integrates reward-related information into context frames upon computing reward expectation by compiling stimulus-reward and context-reward information offered by the amygdala and hippocampus, respectively. Furthermore we suggest that the integration of model-based expectations regarding reward into the value signal is further supported by the efferent of the OFC that reach structures canonical for model-free learning (e.g., the PPTgN, VTA, and VS). (c) 2016 APA, all rights reserved).
-
State-based versus reward-based motivation in younger and older adults.
Worthy, Darrell A; Cooper, Jessica A; Byrne, Kaileigh A; Gorlick, Marissa A; Maddox, W Todd
2014-12-01
Recent decision-making work has focused on a distinction between a habitual, model-free neural system that is motivated toward actions that lead directly to reward and a more computationally demanding goal-directed, model-based system that is motivated toward actions that improve one's future state. In this article, we examine how aging affects motivation toward reward-based versus state-based decision making. Participants performed tasks in which one type of option provided larger immediate rewards but the alternative type of option led to larger rewards on future trials, or improvements in state. We predicted that older adults would show a reduced preference for choices that led to improvements in state and a greater preference for choices that maximized immediate reward. We also predicted that fits from a hybrid reinforcement-learning model would indicate greater model-based strategy use in younger than in older adults. In line with these predictions, older adults selected the options that maximized reward more often than did younger adults in three of the four tasks, and modeling results suggested reduced model-based strategy use. In the task where older adults showed similar behavior to younger adults, our model-fitting results suggested that this was due to the utilization of a win-stay-lose-shift heuristic rather than a more complex model-based strategy. Additionally, within older adults, we found that model-based strategy use was positively correlated with memory measures from our neuropsychological test battery. We suggest that this shift from state-based to reward-based motivation may be due to age related declines in the neural structures needed for more computationally demanding model-based decision making.
-
Stolyarova, Alexandra; Izquierdo, Alicia
2015-01-01
Adolescent behavior is typified by increased risk-taking, reward- and novelty-seeking, as well as an augmented need for social and environmental stimulation. This behavioral phenotype may result from alterations in outcome valuation or reward learning. In the present set of experiments, we directly compared adult and adolescent animals on tasks measuring both of these processes. Additionally, we examined developmental differences in dopamine D1-like receptor (D1R), dopamine D2-like receptor (D2R), and polysialylated neural cell adhesion molecule (PSA-NCAM) expression in animals that were trained on an effortful reward valuation task, given that these proteins play an important role in the functional development of the amygdala-prefrontocortical (PFC) circuit and mesocorticolimbic dopamine system. We found that adolescent animals were not different from adults in appetitive associative learning, but exhibited distinct pattern of responses to differences in outcome values, which was paralleled by an enhanced motivation to invest effort to obtain larger rewards. There were no differences in D2 receptor expression, but D1 receptor expression was significantly reduced in the striatum of animals that had experiences with reward learning during adolescence compared to animals that went through the same experiences in adulthood. We observed increased levels of PSA-NCAM expression in both PFC and amygdala of late adolescents compared to adults that were previously trained on an effortful reward valuation task. PSA-NCAM levels in PFC were strongly and positively associated with high effort/reward (HER) choices in adolescents, but not in adult animals. Increased levels of PSA-NCAM expression in adolescents may index increased structural plasticity and represent a neural correlate of a reward sensitive endophenotype. PMID:25999830
-
A Neural Computational Model of Incentive Salience
Zhang, Jun; Berridge, Kent C.; Tindell, Amy J.; Smith, Kyle S.; Aldridge, J. Wayne
2009-01-01
Incentive salience is a motivational property with ‘magnet-like’ qualities. When attributed to reward-predicting stimuli (cues), incentive salience triggers a pulse of ‘wanting’ and an individual is pulled toward the cues and reward. A key computational question is how incentive salience is generated during a cue re-encounter, which combines both learning and the state of limbic brain mechanisms. Learning processes, such as temporal-difference models, provide one way for stimuli to acquire cached predictive values of rewards. However, empirical data show that subsequent incentive values are also modulated on the fly by dynamic fluctuation in physiological states, altering cached values in ways requiring additional motivation mechanisms. Dynamic modulation of incentive salience for a Pavlovian conditioned stimulus (CS or cue) occurs during certain states, without necessarily requiring (re)learning about the cue. In some cases, dynamic modulation of cue value occurs during states that are quite novel, never having been experienced before, and even prior to experience of the associated unconditioned reward in the new state. Such cases can include novel drug-induced mesolimbic activation and addictive incentive-sensitization, as well as natural appetite states such as salt appetite. Dynamic enhancement specifically raises the incentive salience of an appropriate CS, without necessarily changing that of other CSs. Here we suggest a new computational model that modulates incentive salience by integrating changing physiological states with prior learning. We support the model with behavioral and neurobiological data from empirical tests that demonstrate dynamic elevations in cue-triggered motivation (involving natural salt appetite, and drug-induced intoxication and sensitization). Our data call for a dynamic model of incentive salience, such as presented here. Computational models can adequately capture fluctuations in cue-triggered ‘wanting’ only by incorporating modulation of previously learned values by natural appetite and addiction-related states. PMID:19609350
-
Lu, Xi; Katoh, Takahiko; Chen, Zi; Nagata, Toshiaki; Kitamura, Toshinori
2014-05-15
Text messaging may be excessive and young people may be dependent on it. We distributed the Self-perception of Text-message Dependency Scale (STDS), Hospital Anxiety and Depression Scale (HADS), Temperament and Character Inventory (TCI), and Relationship Questionnaire (RQ) to 223 Japanese university students in a two-wave study, separated by a 5-month interval. The STDS yielded a three-factor structure. The STDS scores across the two measurement occasions were stable across time (except for the Relationship Maintenance subscale). A hierarchical cluster analysis suggested a three-class structure interpreted as Normal Users, Excessive Users, and Dependent Users. Excessive Users and Dependent Users were characterized by a young age at initial mobile phone use, more frequent use of text messaging, higher Novelty Seeking, and better Other-Model patterns of adult attachment. Unlike Excessive Users, Dependent Users were characterized by lower Self-directedness, poorer Self-Model of adult attachment, and higher anxiety and depression. The Excessive Users, but not the Dependent Users, were characterized by high Reward Dependence and Co-operativeness. The present study demonstrated that the STDS has a robust factor structure, good construct validity, and temporal stability (except for Relationship Maintenance subscale); students could be classified into normal, excessive, and Dependent Users of the text messaging; and Dependent Users were characterized by Excessive Use and personality immaturity. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.
-
Overcommitment as a predictor of effort-reward imbalance: evidence from an 8-year follow-up study.
Feldt, Taru; Hyvönen, Katriina; Mäkikangas, Anne; Rantanen, Johanna; Huhtala, Mari; Kinnunen, Ulla
2016-07-01
The effort-reward imbalance (ERI) model includes the personal characteristic of overcommitment (OC) and the job-related characteristics of effort, reward, and ERI, all of which are assumed to play a role in an employee's health and well-being at work. The aim of the present longitudinal study was to shed more light on the dynamics of the ERI model by investigating the basic hypotheses related to the role of OC in the model, ie, to establish whether an employee's OC could be a risk factor for an increased experience of high effort, low reward, and high ERI at work. The study was based on 5-wave, 8-year follow-up data collected among Finnish professionals in 2006 (T1, N=747), 2008 (T2, N=422), 2010 (T3, N=368), 2012 (T4, N=325), and 2014 (T5, N=273). The participants were mostly male (85% at T1) and the majority of them worked in technical fields. OC, effort, reward, and ERI were measured at each time point with the 23-item ERI scale. Three cross-lagged structural equation models (SEM) were estimated and compared by using full information maximum likelihood method: (i) OC predicted later experiences of effort, reward, and ERI (normal causation model), (ii) effort, reward, and ERI predicted later OC (reversed causation model), and (iii) associations in normal causal and reversed causal models were simultaneously valid (reciprocal causation model). The results supported the normal causation model: strong OC predicted later experiences of high effort, low reward and high ERI. High OC is a risk factor for an increased experience of job strain factors; that is, high effort, low reward, and high ERI. Thus, OC is a risk factor not only for an employee's well-being and health but also for an increasing risk for perceiving adverse job strain factors in the working environment.
-
Castro, Daniel C.; Cole, Shannon L.; Berridge, Kent C.
2015-01-01
The study of the neural bases of eating behavior, hunger, and reward has consistently implicated the lateral hypothalamus (LH) and its interactions with mesocorticolimbic circuitry, such as mesolimbic dopamine projections to nucleus accumbens (NAc) and ventral pallidum (VP), in controlling motivation to eat. The NAc and VP play special roles in mediating the hedonic impact (“liking”) and motivational incentive salience (“wanting”) of food rewards, and their interactions with LH help permit regulatory hunger/satiety modulation of food motivation and reward. Here, we review some progress that has been made regarding this circuitry and its functions: the identification of localized anatomical hedonic hotspots within NAc and VP for enhancing hedonic impact; interactions of NAc/VP hedonic hotspots with specific LH signals such as orexin; an anterior-posterior gradient of sites in NAc shell for producing intense appetitive eating vs. intense fearful reactions; and anatomically distributed appetitive functions of dopamine and mu opioid signals in NAc shell and related structures. Such findings help improve our understanding of NAc, VP, and LH interactions in mediating affective and motivation functions, including “liking” and “wanting” for food rewards. PMID:26124708
-
Neural correlates of specific musical anhedonia
Martínez-Molina, Noelia; Mas-Herrero, Ernest; Rodríguez-Fornells, Antoni; Zatorre, Robert J.
2016-01-01
Although music is ubiquitous in human societies, there are some people for whom music holds no reward value despite normal perceptual ability and preserved reward-related responses in other domains. The study of these individuals with specific musical anhedonia may be crucial to understand better the neural correlates underlying musical reward. Previous neuroimaging studies have shown that musically induced pleasure may arise from the interaction between auditory cortical networks and mesolimbic reward networks. If such interaction is critical for music-induced pleasure to emerge, then those individuals who do not experience it should show alterations in the cortical-mesolimbic response. In the current study, we addressed this question using fMRI in three groups of 15 participants, each with different sensitivity to music reward. We demonstrate that the music anhedonic participants showed selective reduction of activity for music in the nucleus accumbens (NAcc), but normal activation levels for a monetary gambling task. Furthermore, this group also exhibited decreased functional connectivity between the right auditory cortex and ventral striatum (including the NAcc). In contrast, individuals with greater than average response to music showed enhanced connectivity between these structures. Thus, our results suggest that specific musical anhedonia may be associated with a reduction in the interplay between the auditory cortex and the subcortical reward network, indicating a pivotal role of this interaction for the enjoyment of music. PMID:27799544
-
Neural activation in the "reward circuit" shows a nonlinear response to facial attractiveness.
Liang, Xiaoyun; Zebrowitz, Leslie A; Zhang, Yi
2010-01-01
Positive behavioral responses to attractive faces have led neuroscientists to investigate underlying neural mechanisms in a "reward circuit" that includes brain regions innervated by dopamine pathways. Using male faces ranging from attractive to extremely unattractive, disfigured ones, this study is the first to demonstrate heightened responses to both rewarding and aversive faces in numerous areas of this putative reward circuit. Parametric analyses employing orthogonal linear and nonlinear regressors revealed positive nonlinear effects in anterior cingulate cortex, lateral orbital frontal cortex (LOFC), striatum (nucleus accumbens, caudate, putamen), and ventral tegmental area, in addition to replicating previously documented linear effects in medial orbital frontal cortex (MOFC) and LOFC and nonlinear effects in amygdala and MOFC. The widespread nonlinear responses are consistent with single cell recordings in animals showing responses to both rewarding and aversive stimuli, and with some human fMRI investigations of non-face stimuli. They indicate that the reward circuit does not process face valence with any simple dissociation of function across structures. Perceiver gender modulated some responses to our male faces: Women showed stronger linear effects, and men showed stronger nonlinear effects, which may have functional implications. Our discovery of nonlinear responses to attractiveness throughout the reward circuit echoes the history of amygdala research: Early work indicated a linear response to threatening stimuli, including faces; later work also revealed a nonlinear response with heightened activation to affectively salient stimuli regardless of valence. The challenge remains to determine how such dual coding influences feelings, such as pleasure and pain, and guides goal-related behavioral responses, such as approach and avoidance.
-
Frank, G K W; Shott, M E; Riederer, J; Pryor, T L
2016-11-01
Anorexia and bulimia nervosa are severe eating disorders that share many behaviors. Structural and functional brain circuits could provide biological links that those disorders have in common. We recruited 77 young adult women, 26 healthy controls, 26 women with anorexia and 25 women with bulimia nervosa. Probabilistic tractography was used to map white matter connectivity strength across taste and food intake regulating brain circuits. An independent multisample greedy equivalence search algorithm tested effective connectivity between those regions during sucrose tasting. Anorexia and bulimia nervosa had greater structural connectivity in pathways between insula, orbitofrontal cortex and ventral striatum, but lower connectivity from orbitofrontal cortex and amygdala to the hypothalamus (P<0.05, corrected for comorbidity, medication and multiple comparisons). Functionally, in controls the hypothalamus drove ventral striatal activity, but in anorexia and bulimia nervosa effective connectivity was directed from anterior cingulate via ventral striatum to the hypothalamus. Across all groups, sweetness perception was predicted by connectivity strength in pathways connecting to the middle orbitofrontal cortex. This study provides evidence that white matter structural as well as effective connectivity within the energy-homeostasis and food reward-regulating circuitry is fundamentally different in anorexia and bulimia nervosa compared with that in controls. In eating disorders, anterior cingulate cognitive-emotional top down control could affect food reward and eating drive, override hypothalamic inputs to the ventral striatum and enable prolonged food restriction.
-
Family Influences on Mania-Relevant Cognitions and Beliefs: A Cognitive Model of Mania and Reward
Chen, Stephen H.; Johnson, Sheri L.
2012-01-01
Objective The present study proposed and tested a cognitive model of mania and reward. Method Undergraduates (N = 284; 68.4% female; mean age = 20.99 years, standard deviation ± 3.37) completed measures of family goal setting and achievement values, personal reward-related beliefs, cognitive symptoms of mania, and risk for mania. Results Correlational analyses and structural equation modeling supported two distinct, but related facets of mania-relevant cognition: stably present reward-related beliefs and state-dependent cognitive symptoms in response to success and positive emotion. Results also indicated that family emphasis on achievement and highly ambitious extrinsic goals were associated with these mania-relevant cognitions. Finally, controlling for other factors, cognitive symptoms in response to success and positive emotion were uniquely associated with lifetime propensity towards mania symptoms. Conclusions Results support the merit of distinguishing between facets of mania-relevant cognition and the importance of the family in shaping both aspects of cognition. PMID:22623269
-
Family influences on mania-relevant cognitions and beliefs: a cognitive model of mania and reward.
Chen, Stephen H; Johnson, Sheri L
2012-07-01
The present study proposed and tested a cognitive model of mania and reward. Undergraduates (N = 284; 68.4% female; mean age = 20.99 years, standard deviation ± 3.37) completed measures of family goal setting and achievement values, personal reward-related beliefs, cognitive symptoms of mania, and risk for mania. Correlational analyses and structural equation modeling supported two distinct, but related facets of mania-relevant cognition: stably present reward-related beliefs and state-dependent cognitive symptoms in response to success and positive emotion. Results also indicated that family emphasis on achievement and highly ambitious extrinsic goals were associated with these mania-relevant cognitions. Finally, controlling for other factors, cognitive symptoms in response to success and positive emotion were uniquely associated with lifetime propensity towards mania symptoms. Results support the merit of distinguishing between facets of mania-relevant cognition and the importance of the family in shaping both aspects of cognition. © 2012 Wiley Periodicals, Inc.
-
White Matter Correlates of Musical Anhedonia: Implications for Evolution of Music.
Loui, Psyche; Patterson, Sean; Sachs, Matthew E; Leung, Yvonne; Zeng, Tima; Przysinda, Emily
2017-01-01
Recent theoretical advances in the evolution of music posit that affective communication is an evolutionary function of music through which the mind and brain are transformed. A rigorous test of this view should entail examining the neuroanatomical mechanisms for affective communication of music, specifically by comparing individual differences in the general population with a special population who lacks specific affective responses to music. Here we compare white matter connectivity in BW, a case with severe musical anhedonia, with a large sample of control subjects who exhibit normal variability in reward sensitivity to music. We show for the first time that structural connectivity within the reward system can predict individual differences in musical reward in a large population, but specific patterns in connectivity between auditory and reward systems are special in an extreme case of specific musical anhedonia. Results support and extend the Mixed Origins of Music theory by identifying multiple neural pathways through which music might operate as an affective signaling system.
-
White Matter Correlates of Musical Anhedonia: Implications for Evolution of Music
Loui, Psyche; Patterson, Sean; Sachs, Matthew E.; Leung, Yvonne; Zeng, Tima; Przysinda, Emily
2017-01-01
Recent theoretical advances in the evolution of music posit that affective communication is an evolutionary function of music through which the mind and brain are transformed. A rigorous test of this view should entail examining the neuroanatomical mechanisms for affective communication of music, specifically by comparing individual differences in the general population with a special population who lacks specific affective responses to music. Here we compare white matter connectivity in BW, a case with severe musical anhedonia, with a large sample of control subjects who exhibit normal variability in reward sensitivity to music. We show for the first time that structural connectivity within the reward system can predict individual differences in musical reward in a large population, but specific patterns in connectivity between auditory and reward systems are special in an extreme case of specific musical anhedonia. Results support and extend the Mixed Origins of Music theory by identifying multiple neural pathways through which music might operate as an affective signaling system. PMID:28993748
-
Chao, Wang; Shuang, Li; Tao, Li; Shanfa, Yu; Junming, Dai
2017-01-01
This study aimed to detect the mediation effect of over-commitment between occupational stress, insomnia, and well-being; and the moderating role of gender, age and job position are also to be analyzed. One thousand six hundred eighteen valid samples were recruited from electronic manufacturing service industry in Hunan Province, China. All the data were collected by selfrated questionnaires after written consent. This paper introduced Effort-Reward- Insomnia-Well-being model, and it was fitted and validated through the structural equation model analysis. The results of single factor correlation analysis indicated that the coefficients between most of the items and dimensions presented statistical significance. The final fitting model had satisfactory global goodness of fit (CMIN/DF=3.99, AGFI=0.926, NNFI=0.950, IFI=0.956, RMSEA=0.043). Both of the measurement model and structural model had acceptable path loadings. Effort associated with insomnia indirectly and related to well-being directly and indirectly; reward could have either directly associated with insomnia and well-being, or indirectly related to them through over-commitment. Covariates as gender, age and position made differences on the association between occupational stress and health outcomes. Over-commitment had the ability to mediate the relationships between effort, reward, and health outcomes, and mediation effect varied from different working conditions and outcomes under different covariates.
-
Leurquin-Sterk, Gil; Ceccarini, Jenny; Crunelle, Cleo Lina; Weerasekera, Akila; de Laat, Bart; Himmelreich, Uwe; Bormans, Guy; Van Laere, Koen
2018-05-01
Converging preclinical evidence links extrastriatal dopamine release and glutamatergic transmission via the metabotropic glutamate receptor 5 (mGluR5) to the rewarding properties of alcohol. To date, human evidence is lacking on how and where in the brain these processes occur. Mesocorticolimbic dopamine release upon intravenous alcohol administration and mGluR5 availability were measured in 11 moderate social drinkers by single-session [ 18 F]fallypride and [ 18 F]FPEB positron emission tomography, respectively. Additionally, baseline and postalcohol glutamate and glutamine levels in the anterior cingulate cortex (ACC) were measured by using proton-magnetic resonance spectroscopy. To investigate differences in reward domains linked to both neurotransmitters, regional imaging data were related to subjective alcohol responses. Alcohol induced significant [ 18 F]fallypride displacement in the prefrontal cortex (PFC), temporal and parietal cortices and thalamus (P < 0.05, corrected for multiple comparisons). Dopamine release in the ACC and orbitofrontal and ventromedial PFCs were correlated with subjective 'liking' and 'wanting' effects (P < 0.05). In contrast, baseline mGluR5 availability was positively correlated with the 'high' effect of alcohol in dorsolateral, ventrolateral and ventromedial PFCs and in the medial temporal lobe, thalamus and caudate nucleus (P < 0.05). Although neither proton-magnetic resonance spectroscopy glutamate nor glutamine levels were affected by alcohol, baseline ACC glutamate levels were negatively associated with the alcohol 'liking' effect (P < 0.003). These data reveal new mechanistic understanding and differential neurobiological underpinnings of the effects of acute alcohol consumption on human behavior. Specifically, prefrontal dopamine release may encode alcohol 'liking' and 'wanting' effects in specific areas underlying value processing and motivation, whereas mGluR5 availability in distinct prefrontal-temporal-subcortical regions is more related to the alcohol 'high' effect. © 2017 Society for the Study of Addiction.
-
Expectancy for food or expectancy for chocolate reveals timing systems for metabolism and reward.
Angeles-Castellanos, M; Salgado-Delgado, R; Rodríguez, K; Buijs, R M; Escobar, C
2008-07-31
The clock gene protein Per 1 (PER1) is expressed in several brain structures and oscillates associated with the suprachiasmatic nucleus (SCN). Restricted feeding schedules (RFS) induce anticipatory activity and impose daily oscillations of c-Fos and clock proteins in brain structures. Daily access to a palatable treat (chocolate) also elicits anticipatory activity and induces c-Fos expression mainly in corticolimbic structures. Here the influence of daily access to food or chocolate was explored by the analysis of the oscillatory patterns of PER1 in hypothalamic and corticolimbic structures. Wistar rats were exposed to RFS or to daily access to chocolate for 3 weeks. Persistence of food or chocolate entrained rhythms was determined 8 days after cessation of the feeding protocols. RFS and chocolate induced a phase shift in PER1 rhythmicity in corticolimbic structures with peak values at zeitgeber time 12 and a higher amplitude in the chocolate group. Both RFS and chocolate groups showed an upregulation of PER1 in the SCN. Food and chocolate entrained rhythms persisted for 8 days in behavior and in PER1 expression in the dorsomedial hypothalamic nucleus, accumbens, prefrontal cortex and central amygdala. The present data demonstrate the existence of different oscillatory systems in the brain that can be activated by entrainment to metabolic stimuli or to reward and suggest the participation of PER1 in both entraining pathways. Persistence and amplification of PER1 oscillations in structures associated with reward suggest that this oscillatory process is fundamental to food addictive behavior.
-
The influences and neural correlates of past and present during gambling in humans.
Sacré, Pierre; Subramanian, Sandya; Kerr, Matthew S D; Kahn, Kevin; Johnson, Matthew A; Bulacio, Juan; González-Martínez, Jorge A; Sarma, Sridevi V; Gale, John T
2017-12-07
During financial decision-making tasks, humans often make "rational" decisions, where they maximize expected reward. However, this rationality may compete with a bias that reflects past outcomes. That is, if one just lost money or won money, this may impact future decisions. It is unclear how past outcomes influence future decisions in humans, and how neural circuits encode present and past information. In this study, six human subjects performed a financial decision-making task while we recorded local field potentials from multiple brain structures. We constructed a model for each subject characterizing bets on each trial as a function of present and past information. The models suggest that some patients are more influenced by previous trial outcomes (i.e., previous return and risk) than others who stick to more fixed decision strategies. In addition, past return and present risk modulated with the activity in the cuneus; while present return and past risk modulated with the activity in the superior temporal gyrus and the angular gyrus, respectively. Our findings suggest that these structures play a role in decision-making beyond their classical functions by incorporating predictions and risks in humans' decision strategy, and provide new insight into how humans link their internal biases to decisions.
-
How musical training affects cognitive development: rhythm, reward and other modulating variables
Miendlarzewska, Ewa A.; Trost, Wiebke J.
2014-01-01
Musical training has recently gained additional interest in education as increasing neuroscientific research demonstrates its positive effects on brain development. Neuroimaging revealed plastic changes in the brains of adult musicians but it is still unclear to what extent they are the product of intensive music training rather than of other factors, such as preexisting biological markers of musicality. In this review, we synthesize a large body of studies demonstrating that benefits of musical training extend beyond the skills it directly aims to train and last well into adulthood. For example, children who undergo musical training have better verbal memory, second language pronunciation accuracy, reading ability and executive functions. Learning to play an instrument as a child may even predict academic performance and IQ in young adulthood. The degree of observed structural and functional adaptation in the brain correlates with intensity and duration of practice. Importantly, the effects on cognitive development depend on the timing of musical initiation due to sensitive periods during development, as well as on several other modulating variables. Notably, we point to motivation, reward and social context of musical education, which are important yet neglected factors affecting the long-term benefits of musical training. Further, we introduce the notion of rhythmic entrainment and suggest that it may represent a mechanism supporting learning and development of executive functions. It also hones temporal processing and orienting of attention in time that may underlie enhancements observed in reading and verbal memory. We conclude that musical training uniquely engenders near and far transfer effects, preparing a foundation for a range of skills, and thus fostering cognitive development. PMID:24672420
-
ERIC Educational Resources Information Center
Hayden, F. Gregory
1980-01-01
By algebraically defining a school finance structure as a total budget system, partial derivatives can be used to find the kinds of rewards, incentives, and distributions the structure defines for individual districts and among districts. Equity concerns can also be answered. (Author/IRT)
-
ERIC Educational Resources Information Center
Schrank, Harris T.; Waring, Joan M.
Business firms are an integral part of the age stratification structure of society. Although the age structures of people and roles within the organization are dynamic, these structures yield a fairly stable strata in which norms exist to suggest the various roles expected of certain persons. Those in roles with greater financial rewards, power,…
-
Food portion size and energy density evoke different patterns of brain activation in children12
Fearnbach, S Nicole; Wilson, Stephen J; Fisher, Jennifer O; Savage, Jennifer S; Rolls, Barbara J; Keller, Kathleen L
2017-01-01
Background: Large portions of food promote intake, but the mechanisms that drive this effect are unclear. Previous neuroimaging studies have identified the brain-reward and decision-making systems that are involved in the response to the energy density (ED) (kilocalories per gram) of foods, but few studies have examined the brain response to the food portion size (PS). Objective: We used functional MRI (fMRI) to determine the brain response to food images that differed in PSs (large and small) and ED (high and low). Design: Block-design fMRI was used to assess the blood oxygen level–dependent (BOLD) response to images in 36 children (7–10 y old; girls: 50%), which was tested after a 2-h fast. Pre-fMRI fullness and liking were rated on visual analog scales. A whole-brain cluster-corrected analysis was used to compare BOLD activation for main effects of the PS, ED, and their interaction. Secondary analyses were used to associate BOLD contrast values with appetitive traits and laboratory intake from meals for which the portions of all foods were increased. Results: Compared with small-PS cues, large-PS cues were associated with decreased activation in the inferior frontal gyrus (P < 0.01). Compared with low-ED cues, high-ED cues were associated with increased activation in multiple regions (e.g., in the caudate, cingulate, and precentral gyrus) and decreased activation in the insula and superior temporal gyrus (P < 0.01 for all). A PS × ED interaction was shown in the superior temporal gyrus (P < 0.01). BOLD contrast values for high-ED cues compared with low-ED cues in the insula, declive, and precentral gyrus were negatively related to appetitive traits (P < 0.05). There were no associations between the brain response to the PS and either appetitive traits or intake. Conclusions: Cues regarding food PS may be processed in the lateral prefrontal cortex, which is a region that is implicated in cognitive control, whereas ED activates multiple areas involved in sensory and reward processing. Possible implications include the development of interventions that target decision-making and reward systems differently to moderate overeating. PMID:27881393
-
The Brazilian version of the effort-reward imbalance questionnaire to assess job stress.
Chor, Dóra; Werneck, Guilherme Loureiro; Faerstein, Eduardo; Alves, Márcia Guimarães de Mello; Rotenberg, Lúcia
2008-01-01
The effort-reward imbalance (ERI) model has been used to assess the health impact of job stress. We aimed at describing the cross-cultural adaptation of the ERI questionnaire into Portuguese and some psychometric properties, in particular internal consistency, test-retest reliability, and factorial structure. We developed a Brazilian version of the ERI using a back-translation method and tested its reliability. The test-retest reliability study was conducted with 111 health workers and University staff. The current analyses are based on 89 participants, after exclusion of those with missing data. Reproducibility (interclass correlation coefficients) for the "effort", "'reward", and "'overcommitment"' dimensions of the scale was estimated at 0.76, 0.86, and 0.78, respectively. Internal consistency (Cronbach's alpha) estimates for these same dimensions were 0.68, 0.78, and 0.78, respectively. The exploratory factorial structure was fairly consistent with the model's theoretical components. We conclude that the results of this study represent the first evidence in favor of the application of the Brazilian Portuguese version of the ERI scale in health research in populations with similar socioeconomic characteristics.
-
Chronic motivational state interacts with task reward structure in dynamic decision-making.
Cooper, Jessica A; Worthy, Darrell A; Maddox, W Todd
2015-12-01
Research distinguishes between a habitual, model-free system motivated toward immediately rewarding actions, and a goal-directed, model-based system motivated toward actions that improve future state. We examined the balance of processing in these two systems during state-based decision-making. We tested a regulatory fit hypothesis (Maddox & Markman, 2010) that predicts that global trait motivation affects the balance of habitual- vs. goal-directed processing but only through its interaction with the task framing as gain-maximization or loss-minimization. We found support for the hypothesis that a match between an individual's chronic motivational state and the task framing enhances goal-directed processing, and thus state-based decision-making. Specifically, chronic promotion-focused individuals under gain-maximization and chronic prevention-focused individuals under loss-minimization both showed enhanced state-based decision-making. Computational modeling indicates that individuals in a match between global chronic motivational state and local task reward structure engaged more goal-directed processing, whereas those in a mismatch engaged more habitual processing. Copyright © 2015 Elsevier Inc. All rights reserved.
-
Yao, Yuan-Wei; Liu, Lu; Ma, Shan-Shan; Shi, Xin-Hui; Zhou, Nan; Zhang, Jin-Tao; Potenza, Marc N
2017-12-01
This meta-analytic study aimed to identify the common and specific neural alterations in Internet gaming disorder (IGD) across different domains and modalities. Two separate meta-analyses for functional neural activation and gray-matter volume were conducted. Sub-meta-analyses for the domains of reward, cold-executive, and hot-executive functions were also performed, respectively. IGD subjects, compared with healthy controls, showed: (1) hyperactivation in the anterior and posterior cingulate cortices, caudate, posterior inferior frontal gyrus (IFG), which were mainly associated with studies measuring reward and cold-executive functions; and, (2) hypoactivation in the anterior IFG in relation to hot-executive function, the posterior insula, somatomotor and somatosensory cortices in relation to reward function. Furthermore, IGD subjects showed reduced gray-matter volume in the anterior cingulate, orbitofrontal, dorsolateral prefrontal, and premotor cortices. These findings suggest that IGD is associated with both functional and structural neural alterations in fronto-striatal and fronto-cingulate regions. Moreover, multi-domain assessments capture different aspects of neural alterations in IGD, which may be helpful for developing effective interventions targeting specific functions. Copyright © 2017 Elsevier Ltd. All rights reserved.
-
[Factor structure of the German version of the BIS/BAS Scales in a population-based sample].
Müller, A; Smits, D; Claes, L; de Zwaan, M
2013-02-01
The Behavioural Inhibition System/Behavioural Activation System Scale (BIS/BAS-Scales) developed by Carver and White 1 is a self-rating instrument to assess the dispositional sensitivity to punishment and reward. The present work aims to examine the factor structure of the German version of the BIS/BAS-Scales. In a large German population-based sample (n = 1881) the model fit of several factor models was tested by using confirmatory factor analyses. The best model fit was found for the 5-factor model with two BIS (anxiety, fear) and three BAS (drive, reward responsiveness, fun seeking) scales, whereas the BIS-fear, the BAS-reward responsiveness, and the BAS-fun seeking subscales showed low internal consistency. The BIS/BAS scales were negatively correlated with age, and women reported higher BIS subscale scores than men. Confirmatory factor analyses suggest a 5-factor model. However, due to the low internal reliability of some of the subscales the use of this model is questionable. © Georg Thieme Verlag KG Stuttgart · New York.
-
Sauer, James D; Drummond, Aaron; Nova, Natalie
2015-09-01
The potential influence of video game violence on real-world aggression has generated considerable public and scientific interest. Some previous research suggests that playing violent video games can increase postgame aggression. The generalized aggression model (GAM) attributes this to the generalized activation of aggressive schemata. However, it is unclear whether game mechanics that contextualize and encourage or inhibit in-game violence moderate this relationship. Thus, we examined the effects of reward structures and narrative context in a violent video game on in-game and postgame aggression. Contrary to GAM-based predictions, our manipulations differentially affected in-game and postgame aggression. Reward structures selectively affected in-game aggression, whereas narrative context selectively affected postgame aggression. Players who enacted in-game violence through a heroic character exhibited less postgame aggression than players who enacted comparable levels of in-game violence through an antiheroic character. Effects were not attributable to self-activation or character-identification mechanisms, but were consistent with social-cognitive context effects on the interpretation of behavior. These results contradict the GAM's assertion that violent video games affect aggression through a generalized activation mechanism. From an applied perspective, consumer choices may be aided by considering not just game content, but the context in which content is portrayed. (c) 2015 APA, all rights reserved).
-
Normalized value coding explains dynamic adaptation in the human valuation process.
Khaw, Mel W; Glimcher, Paul W; Louie, Kenway
2017-11-28
The notion of subjective value is central to choice theories in ecology, economics, and psychology, serving as an integrated decision variable by which options are compared. Subjective value is often assumed to be an absolute quantity, determined in a static manner by the properties of an individual option. Recent neurobiological studies, however, have shown that neural value coding dynamically adapts to the statistics of the recent reward environment, introducing an intrinsic temporal context dependence into the neural representation of value. Whether valuation exhibits this kind of dynamic adaptation at the behavioral level is unknown. Here, we show that the valuation process in human subjects adapts to the history of previous values, with current valuations varying inversely with the average value of recently observed items. The dynamics of this adaptive valuation are captured by divisive normalization, linking these temporal context effects to spatial context effects in decision making as well as spatial and temporal context effects in perception. These findings suggest that adaptation is a universal feature of neural information processing and offer a unifying explanation for contextual phenomena in fields ranging from visual psychophysics to economic choice.
-
Sonuga-Barke, Edmund J S
2014-02-01
Young people with conduct disorder often experience histories of psychosocial adversity and socioeconomic insecurity. For these individuals, real-world future outcomes are not only delayed in their delivery but also highly uncertain. Under such circumstances, accentuated time preference (extreme favoring of the present over the future) is a rational response to the everyday reality of social and economic transactions. Building on this observation, the author sets out the hypothesis that the exaggerated temporal discounting displayed by individuals with conduct disorder reported by White et al. (2014) is an adaptation to chronic exposure to psychosocial insecurity during development. The author postulates that this adaptation leads to (a) a decision-making bias whereby delay and uncertainty are coded as inseparable characteristics of choice outcomes and/or (b) reprogramming of the brain networks regulating intertemporal decision making. Future research could explore the putative role of environmental exposures to adversity in the development of exaggerated temporal discounting in conduct disorder as well as the mediating role of putative cognitive and neurobiological adaptations.
-
Novelty and Inductive Generalization in Human Reinforcement Learning.
Gershman, Samuel J; Niv, Yael
2015-07-01
In reinforcement learning (RL), a decision maker searching for the most rewarding option is often faced with the question: What is the value of an option that has never been tried before? One way to frame this question is as an inductive problem: How can I generalize my previous experience with one set of options to a novel option? We show how hierarchical Bayesian inference can be used to solve this problem, and we describe an equivalence between the Bayesian model and temporal difference learning algorithms that have been proposed as models of RL in humans and animals. According to our view, the search for the best option is guided by abstract knowledge about the relationships between different options in an environment, resulting in greater search efficiency compared to traditional RL algorithms previously applied to human cognition. In two behavioral experiments, we test several predictions of our model, providing evidence that humans learn and exploit structured inductive knowledge to make predictions about novel options. In light of this model, we suggest a new interpretation of dopaminergic responses to novelty. Copyright © 2015 Cognitive Science Society, Inc.
-
Novelty and Inductive Generalization in Human Reinforcement Learning
Gershman, Samuel J.; Niv, Yael
2015-01-01
In reinforcement learning, a decision maker searching for the most rewarding option is often faced with the question: what is the value of an option that has never been tried before? One way to frame this question is as an inductive problem: how can I generalize my previous experience with one set of options to a novel option? We show how hierarchical Bayesian inference can be used to solve this problem, and describe an equivalence between the Bayesian model and temporal difference learning algorithms that have been proposed as models of reinforcement learning in humans and animals. According to our view, the search for the best option is guided by abstract knowledge about the relationships between different options in an environment, resulting in greater search efficiency compared to traditional reinforcement learning algorithms previously applied to human cognition. In two behavioral experiments, we test several predictions of our model, providing evidence that humans learn and exploit structured inductive knowledge to make predictions about novel options. In light of this model, we suggest a new interpretation of dopaminergic responses to novelty. PMID:25808176
-
Osaka, Kengo; Toriumi, Fujio; Sugawara, Toshihauru
2017-01-01
Social networking services (SNSs) are widely used as communicative tools for a variety of purposes. SNSs rely on the users' individual activities associated with some cost and effort, and thus it is not known why users voluntarily continue to participate in SNSs. Because the structures of SNSs are similar to that of the public goods (PG) game, some studies have focused on why voluntary activities emerge as an optimal strategy by modifying the PG game. However, their models do not include direct reciprocity between users, even though reciprocity is a key mechanism that evolves and sustains cooperation in human society. We developed an abstract SNS model called the reciprocity rewards and meta-rewards games that include direct reciprocity by extending the existing models. Then, we investigated how direct reciprocity in an SNS facilitates cooperation that corresponds to participation in SNS by posting articles and comments and how the structure of the networks of users exerts an influence on the strategies of users using the reciprocity rewards game. We run reciprocity rewards games on various complex networks and an instance network of Facebook and found that two types of stable cooperation emerged. First, reciprocity slightly improves the rate of cooperation in complete graphs but the improvement is insignificant because of the instability of cooperation. However, this instability can be avoided by making two assumptions: high degree of fun, i.e. articles are read with high probability, and different attitudes to reciprocal and non-reciprocal agents. We then propose the concept of half free riders to explain what strategy sustains cooperation-dominant situations. Second, we indicate that a certain WS network structure affects users' optimal strategy and facilitates stable cooperation without any extra assumptions. We give a detailed analysis of the different characteristics of the two types of cooperation-dominant situations and the effect of the memory of reciprocal agents on cooperation.
-
Jansen, Heiko T; Sergeeva, Anna; Stark, Gemaine; Sorg, Barbara A
2012-05-01
A unique extra-suprachiasmatic nucleus (SCN) oscillator, operating independently of the light-entrainable oscillator, has been hypothesized to generate feeding and drug-related rhythms. To test the validity of this hypothesis, sham-lesioned (Sham) and SCN-lesioned (SCNx) rats were housed in constant dim-red illumination (LL(red)) and received a daily cocaine injection every 24 h for 7 d (Experiment 1). In a second experiment, rats underwent 3-h daily restricted feeding (RF) followed 12 d later by the addition of daily cocaine injections given every 25 h in combination with RF until the two schedules were in antiphase. In both experiments, body temperature and total activity were monitored continuously. Results from Experiment 1 revealed that cocaine, but not saline, injections produced anticipatory increases in temperature and activity in SCNx and Sham rats. Following withdrawal from cocaine, free-running temperature rhythms persisted for 2-10 d in SCNx rats. In Experiment 2, robust anticipatory increases in temperature and activity were associated with RF and cocaine injections; however, the feeding periodicity (23.9 h) predominated over the cocaine periodicity. During drug withdrawal, the authors observed two free-running rhythms of temperature and activity that persisted for >14 d in both Sham and SCNx rats. The periods of the free-running rhythms were similar to the feeding entrainment (period = 23.7 and 24.0 h, respectively) and drug entrainment (period = 25.7 and 26.1 h, respectively). Also during withdrawal, the normally close correlation between activity and temperature was greatly disrupted in Sham and SCNx rats. Taken together, these results do not support the existence of a single oscillator mediating the rewarding properties of both food and cocaine. Rather, they suggest that these two highly rewarding behaviors can be temporally isolated, especially during drug withdrawal. Under stable dual-entrainment conditions, food reward appears to exhibit a slightly greater circadian influence than drug reward. The ability to generate free-running temperature rhythms of different frequencies following combined food and drug exposures could reflect a state of internal desynchrony that may contribute to the addiction process and drug relapse.
-
Larsen, Tobias; Collette, Sven; Tyszka, Julian M.; Seymour, Ben; O'Doherty, John P.
2015-01-01
The role of neurons in the substantia nigra (SN) and ventral tegmental area (VTA) of the midbrain in contributing to the elicitation of reward prediction errors during appetitive learning has been well established. Less is known about the differential contribution of these midbrain regions to appetitive versus aversive learning, especially in humans. Here we scanned human participants with high-resolution fMRI focused on the SN and VTA while they participated in a sequential Pavlovian conditioning paradigm involving an appetitive outcome (a pleasant juice), as well as an aversive outcome (an unpleasant bitter and salty flavor). We found a degree of regional specialization within the SN: Whereas a region of ventromedial SN correlated with a temporal difference reward prediction error during appetitive Pavlovian learning, a dorsolateral area correlated instead with an aversive expected value signal in response to the most distal cue, and to a reward prediction error in response to the most proximal cue to the aversive outcome. Furthermore, participants' affective reactions to both the appetitive and aversive conditioned stimuli more than 1 year after the fMRI experiment was conducted correlated with activation in the ventromedial and dorsolateral SN obtained during the experiment, respectively. These findings suggest that, whereas the human ventromedial SN contributes to long-term learning about rewards, the dorsolateral SN may be particularly important for long-term learning in aversive contexts. SIGNIFICANCE STATEMENT The role of the substantia nigra (SN) and ventral tegmental area (VTA) in appetitive learning is well established, but less is known about their contribution to aversive compared with appetitive learning, especially in humans. We used high-resolution fMRI to measure activity in the SN and VTA while participants underwent higher-order Pavlovian learning. We found a regional specialization within the SN: a ventromedial area was selectively engaged during appetitive learning, and a dorsolateral area during aversive learning. Activity in these areas predicted affective reactions to appetitive and aversive conditioned stimuli over 1 year later. These findings suggest that, whereas the human ventromedial SN contributes to long-term learning about rewards, the dorsolateral SN may be particularly important for long-term learning in aversive contexts. PMID:26490862
-
Haight, Joshua L.; Fraser, Kurt M.; Akil, Huda; Flagel, Shelly B.
2015-01-01
Recently evidence has emerged suggesting a role for the paraventricular nucleus of the thalamus (PVT) in the processing of reward-associated cues. However, the specific role of the PVT in these processes has yet to be elucidated. Here we use an animal model that captures individual variation in response to discrete reward-associated cues to further assess the role of the PVT in stimulus-reward learning. When rats are exposed to a Pavlovian conditioning paradigm, wherein a discrete cue predicts food reward, two distinct conditioned responses emerge. Some rats, termed sign-trackers, approach and manipulate the cue; whereas others, termed goal-trackers, approach the location of reward delivery upon cue presentation. For both sign- and goal-trackers the cue is a predictor; but only for sign-trackers is it also an incentive stimulus. We investigated the role of the PVT in the acquisition and expression of these conditioned responses using an excitotoxic lesion. Results indicate that PVT lesions prior to acquisition amplify the differences between phenotypes—increasing sign-tracking and attenuating goal-tracking behavior. Lesions of the PVT after rats had acquired their respective conditioned responses also attenuated the expression of the goal-tracking response, and increased the sign-tracking response, but did so selectively in goal-trackers. These results suggest that the PVT acts to suppress the attribution of incentive salience to reward cues, as disruption of the functional activity within this structure enhances the tendency to sign-track. PMID:26228683
-
Weisser, Fabia B; Bristowe, Katherine; Jackson, Diana
2015-09-01
Family caregivers of people with Motor Neurone Disease/Amyotrophic Lateral Sclerosis, an incurable, mostly rapidly fatal neurodegenerative disease, face many challenges. Although there is considerable research on caregiver burden in Motor Neurone Disease/Amyotrophic Lateral Sclerosis, there is less knowledge of the positive aspects of caring. To explore the experiences of family caregivers of people with Motor Neurone Disease/Amyotrophic Lateral Sclerosis, specifically the relationship between positive and negative experiences of caring, and to identify possible ways to better support these caregivers. Secondary thematic analysis of 24 semi-structured qualitative interviews conducted longitudinally with 10 family caregivers. Interviews explored rewarding and unrewarding aspects of caring. Themes emerged around burden, needs, rewards and resilience. Resilience included getting active, retaining perspective and living for the moment. Burden was multifaceted, including social burden, responsibility, advocacy, ambivalence, guilt and struggling with acceptance. Rewards included being helped and 'ticking along'. Needs were multifaceted, including social, practical and psychological needs. The four main themes were interrelated. A model of coping was developed, integrating resilience (active/positive), burden (active/negative), needs (passive/negative) and reward (passive/positive). Burden, resilience, needs and rewards are interrelated. Caregivers' ability to cope with caring for a person with Motor Neurone Disease/Amyotrophic Lateral Sclerosis oscillates between positive and negative aspects of caring, being at times active, at times passive. Coping is a non-linear process, oscillating between different states of mind. The proposed model could enable clinicians to better understand the caregiver experience, help family caregivers foster resilience and identify rewards, and develop appropriate individualised caregiver support plans. © The Author(s) 2015.
-
Macoveanu, Julian; Fisher, Patrick M; Haahr, Mette E; Frokjaer, Vibe G; Knudsen, Gitte M; Siebner, Hartwig R
2014-10-01
Selective serotonin reuptake inhibitors (SSRIs) such as fluoxetine are commonly prescribed antidepressant drugs targeting the dysfunctional serotonin (5-HT) system, yet little is known about the functional effects of prolonged serotonin reuptake inhibition in healthy individuals. Here we used functional MRI (fMRI) to investigate how a three-week fluoxetine intervention influences neural activity related to risk taking and reward processing. Employing a double-blinded parallel-group design, 29 healthy young males were randomly assigned to receive 3 weeks of a daily dose of 40 mg fluoxetine or placebo. Participants underwent task-related fMRI prior to and after the three-week intervention while performing a card gambling task. The task required participants to choose between two decks of cards. Choices were associated with different risk levels and potential reward magnitudes. Relative to placebo, the SSRI intervention did not alter individual risk-choice preferences, but modified neural activity during decision-making and reward processing: During the choice phase, SSRI reduced the neural response to increasing risk in lateral orbitofrontal cortex, a key structure for value-based decision-making. During the outcome phase, a midbrain region showed an independent decrease in the responsiveness to rewarding outcomes. This midbrain cluster included the raphe nuclei from which serotonergic modulatory projections originate to both cortical and subcortical regions. The findings corroborate the involvement of the normally functioning 5HT-system in decision-making under risk and processing of monetary rewards. The data suggest that prolonged SSRI treatment might reduce emotional engagement by reducing the impact of risk during decision-making or the impact of reward during outcome evaluation. Copyright © 2014 Elsevier Inc. All rights reserved.
-
Sex differences in the influence of body mass index on anatomical architecture of brain networks.
Gupta, A; Mayer, E A; Hamadani, K; Bhatt, R; Fling, C; Alaverdyan, M; Torgerson, C; Ashe-McNalley, C; Van Horn, J D; Naliboff, B; Tillisch, K; Sanmiguel, C P; Labus, J S
2017-08-01
The brain has a central role in regulating ingestive behavior in obesity. Analogous to addiction behaviors, an imbalance in the processing of rewarding and salient stimuli results in maladaptive eating behaviors that override homeostatic needs. We performed network analysis based on graph theory to examine the association between body mass index (BMI) and network measures of integrity, information flow and global communication (centrality) in reward, salience and sensorimotor regions and to identify sex-related differences in these parameters. Structural and diffusion tensor imaging were obtained in a sample of 124 individuals (61 males and 63 females). Graph theory was applied to calculate anatomical network properties (centrality) for regions of the reward, salience and sensorimotor networks. General linear models with linear contrasts were performed to test for BMI and sex-related differences in measures of centrality, while controlling for age. In both males and females, individuals with high BMI (obese and overweight) had greater anatomical centrality (greater connectivity) of reward (putamen) and salience (anterior insula) network regions. Sex differences were observed both in individuals with normal and elevated BMI. In individuals with high BMI, females compared to males showed greater centrality in reward (amygdala, hippocampus and nucleus accumbens) and salience (anterior mid-cingulate cortex) regions, while males compared to females had greater centrality in reward (putamen) and sensorimotor (posterior insula) regions. In individuals with increased BMI, reward, salience and sensorimotor network regions are susceptible to topological restructuring in a sex-related manner. These findings highlight the influence of these regions on integrative processing of food-related stimuli and increased ingestive behavior in obesity, or in the influence of hedonic ingestion on brain topological restructuring. The observed sex differences emphasize the importance of considering sex differences in obesity pathophysiology.
-
Sex Differences in the Influence of Body Mass Index on Anatomical Architecture of Brain Networks
Gupta, Arpana; Mayer, Emeran A.; Hamadani, Kareem; Bhatt, Ravi; Fling, Connor; Alaverdyan, Mher; Torgenson, Carinna; Ashe-McNalley, Cody; Van Horn, John D; Naliboff, Bruce; Tillisch, Kirsten; Sanmiguel, Claudia P.; Labus, Jennifer S.
2017-01-01
Background/Objective The brain plays a central role in regulating ingestive behavior in obesity. Analogous to addiction behaviors, an imbalance in the processing of rewarding and salient stimuli results in maladaptive eating behaviors that override homeostatic needs. We performed network analysis based on graph theory to examine the association between body mass index (BMI) and network measures of integrity, information flow, and global communication (centrality) in reward, salience and sensorimotor regions, and to identify sex-related differences in these parameters. Subjects/Methods Structural and diffusion tensor imaging were obtained in a sample of 124 individuals (61 males and 63 females). Graph theory was applied to calculate anatomical network properties (centrality) for regions of the reward, salience, and sensorimotor networks. General linear models with linear contrasts were performed to test for BMI and sex-related differences in measures of centrality, while controlling for age. Results In both males and females, individuals with high BMI (obese and overweight) had greater anatomical centrality (greater connectivity) of reward (putamen) and salience (anterior insula) network regions. Sex differences were observed both in individuals with normal and elevated BMI. In individuals with high BMI, females compared to males showed greater centrality in reward (amygdala, hippocampus, nucleus accumbens) and salience (anterior mid cingulate cortex) regions, while males compared to females had greater centrality in reward (putamen) and sensorimotor (posterior insula) regions. Conclusions In individuals with increased BMI, reward, salience, and sensorimotor network regions are susceptible to topological restructuring in a sex related manner. These findings highlight the influence of these regions on integrative processing of food-related stimuli and increased ingestive behavior in obesity, or in the influence of hedonic ingestion on brain topological restructuring. The observed sex differences emphasize the importance of considering sex differences in obesity pathophysiology. PMID:28360430
-
Tukey, David S; Lee, Michelle; Xu, Duo; Eberle, Sarah E; Goffer, Yossef; Manders, Toby R; Ziff, Edward B; Wang, Jing
2013-07-09
Pain and natural rewards such as food elicit different behavioral effects. Both pain and rewards, however, have been shown to alter synaptic activities in the nucleus accumbens (NAc), a key component of the brain reward system. Mechanisms by which external stimuli regulate plasticity at NAc synapses are largely unexplored. Medium spiny neurons (MSNs) from the NAc receive excitatory glutamatergic inputs and modulatory dopaminergic and cholinergic inputs from a variety of cortical and subcortical structures. Glutamate inputs to the NAc arise primarily from prefrontal cortex, thalamus, amygdala, and hippocampus, and different glutamate projections provide distinct synaptic and ultimately behavioral functions. The family of vesicular glutamate transporters (VGLUTs 1-3) plays a key role in the uploading of glutamate into synaptic vesicles. VGLUT1-3 isoforms have distinct expression patterns in the brain, but the effects of external stimuli on their expression patterns have not been studied. In this study, we use a sucrose self-administration paradigm for natural rewards, and spared nerve injury (SNI) model for chronic pain. We examine the levels of VGLUTs (1-3) in synaptoneurosomes of the NAc in these two behavioral models. We find that chronic pain leads to a decrease of VGLUT1, likely reflecting decreased projections from the cortex. Pain also decreases VGLUT3 levels, likely representing a decrease in projections from GABAergic, serotonergic, and/or cholinergic interneurons. In contrast, chronic consumption of sucrose increases VGLUT3 in the NAc, possibly reflecting an increase from these interneuron projections. Our study shows that natural rewards and pain have distinct effects on the VGLUT expression pattern in the NAc, indicating that glutamate inputs to the NAc are differentially modulated by rewards and pain.
-
A neuronal model of a global workspace in effortful cognitive tasks.
Dehaene, S; Kerszberg, M; Changeux, J P
1998-11-24
A minimal hypothesis is proposed concerning the brain processes underlying effortful tasks. It distinguishes two main computational spaces: a unique global workspace composed of distributed and heavily interconnected neurons with long-range axons, and a set of specialized and modular perceptual, motor, memory, evaluative, and attentional processors. Workspace neurons are mobilized in effortful tasks for which the specialized processors do not suffice. They selectively mobilize or suppress, through descending connections, the contribution of specific processor neurons. In the course of task performance, workspace neurons become spontaneously coactivated, forming discrete though variable spatio-temporal patterns subject to modulation by vigilance signals and to selection by reward signals. A computer simulation of the Stroop task shows workspace activation to increase during acquisition of a novel task, effortful execution, and after errors. We outline predictions for spatio-temporal activation patterns during brain imaging, particularly about the contribution of dorsolateral prefrontal cortex and anterior cingulate to the workspace.
-
Auditory Gap-in-Noise Detection Behavior in Ferrets and Humans
2015-01-01
The precise encoding of temporal features of auditory stimuli by the mammalian auditory system is critical to the perception of biologically important sounds, including vocalizations, speech, and music. In this study, auditory gap-detection behavior was evaluated in adult pigmented ferrets (Mustelid putorius furo) using bandpassed stimuli designed to widely sample the ferret’s behavioral and physiological audiogram. Animals were tested under positive operant conditioning, with psychometric functions constructed in response to gap-in-noise lengths ranging from 3 to 270 ms. Using a modified version of this gap-detection task, with the same stimulus frequency parameters, we also tested a cohort of normal-hearing human subjects. Gap-detection thresholds were computed from psychometric curves transformed according to signal detection theory, revealing that for both ferrets and humans, detection sensitivity was worse for silent gaps embedded within low-frequency noise compared with high-frequency or broadband stimuli. Additional psychometric function analysis of ferret behavior indicated effects of stimulus spectral content on aspects of behavioral performance related to decision-making processes, with animals displaying improved sensitivity for broadband gap-in-noise detection. Reaction times derived from unconditioned head-orienting data and the time from stimulus onset to reward spout activation varied with the stimulus frequency content and gap length, as well as the approach-to-target choice and reward location. The present study represents a comprehensive evaluation of gap-detection behavior in ferrets, while similarities in performance with our human subjects confirm the use of the ferret as an appropriate model of temporal processing. PMID:26052794
-
Prospective Coding by Spiking Neurons
Brea, Johanni; Gaál, Alexisz Tamás; Senn, Walter
2016-01-01
Animals learn to make predictions, such as associating the sound of a bell with upcoming feeding or predicting a movement that a motor command is eliciting. How predictions are realized on the neuronal level and what plasticity rule underlies their learning is not well understood. Here we propose a biologically plausible synaptic plasticity rule to learn predictions on a single neuron level on a timescale of seconds. The learning rule allows a spiking two-compartment neuron to match its current firing rate to its own expected future discounted firing rate. For instance, if an originally neutral event is repeatedly followed by an event that elevates the firing rate of a neuron, the originally neutral event will eventually also elevate the neuron’s firing rate. The plasticity rule is a form of spike timing dependent plasticity in which a presynaptic spike followed by a postsynaptic spike leads to potentiation. Even if the plasticity window has a width of 20 milliseconds, associations on the time scale of seconds can be learned. We illustrate prospective coding with three examples: learning to predict a time varying input, learning to predict the next stimulus in a delayed paired-associate task and learning with a recurrent network to reproduce a temporally compressed version of a sequence. We discuss the potential role of the learning mechanism in classical trace conditioning. In the special case that the signal to be predicted encodes reward, the neuron learns to predict the discounted future reward and learning is closely related to the temporal difference learning algorithm TD(λ). PMID:27341100
-
Neural correlates of reward and loss sensitivity in psychopathy.
Pujara, Maia; Motzkin, Julian C; Newman, Joseph P; Kiehl, Kent A; Koenigs, Michael
2014-06-01
Psychopathy is a personality disorder associated with callous and impulsive behavior and criminal recidivism. It has long been theorized that psychopaths have deficits in processing reward and punishment. Here, we use structural and functional magnetic resonance imaging to examine the neural correlates of reward and loss sensitivity in a group of criminal psychopaths. Forty-one adult male prison inmates (n = 18 psychopaths and n = 23 non-psychopaths) completed a functional magnetic resonance imaging task involving the gain or loss of money. Across the entire sample of participants, monetary gains elicited robust activation within the ventral striatum (VS). Although psychopaths and non-psychopaths did not significantly differ with respect to overall levels of VS response to reward vs loss, we observed significantly different correlations between VS responses and psychopathy severity within each group. Volumetric analyses of striatal subregions revealed a similar pattern of correlations, specifically for the right accumbens area within VS. In a separate sample of inmates (n = 93 psychopaths and n = 117 non-psychopaths) who completed a self-report measure of appetitive motivation, we again found that the correlation with psychopathy severity differed between groups. These convergent results offer novel insight into the neural substrates of reward and loss processing in psychopathy. © The Author (2013). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
de Clauser, Larissa; Kasper, Hansjörg; Schwab, Martin E.
2016-01-01
Motor skills represent high-precision movements performed at optimal speed and accuracy. Such motor skills are learned with practice over time. Besides practice, effects of motivation have also been shown to influence speed and accuracy of movements, suggesting that fast movements are performed to maximize gained reward over time as noted in previous studies. In rodents, skilled motor performance has been successfully modeled with the skilled grasping task, in which animals use their forepaw to grasp for sugar pellet rewards through a narrow window. Using sugar pellets, the skilled grasping task is inherently tied to motivation processes. In the present study, we performed three experiments modulating animals’ motivation during skilled grasping by changing the motivational state, presenting different reward value ratios, and displaying Pavlovian stimuli. We found in all three studies that motivation affected the speed of skilled grasping movements, with the strongest effects seen due to motivational state and reward value. Furthermore, accuracy of the movement, measured in success rate, showed a strong dependence on motivational state as well. Pavlovian cues had only minor effects on skilled grasping, but results indicate an inverse Pavlovian-instrumental transfer effect on movement speed. These findings have broad implications considering the increasing use of skilled grasping in studies of motor system structure, function, and recovery after injuries. PMID:27194796
-
Enhancing Student Experiential Learning with Structured Interviews
ERIC Educational Resources Information Center
Cornell, Robert M.; Johnson, Carol B.; Schwartz, William C., Jr.
2013-01-01
Learning through experience can be rewarding but intimidating. To maximize the benefits of experiential learning assignments, students need to have confidence in their abilities. The authors report how a structured-interview instrument effectively facilitated experiential learning for accounting students without extensive content-specific…
-
Stress hormones predict hyperbolic time-discount rates six months later in adults.
Takahashi, Taiki; Shinada, Mizuho; Inukai, Keigo; Tanida, Shigehito; Takahashi, Chisato; Mifune, Nobuhiro; Takagishi, Haruto; Horita, Yutaka; Hashimoto, Hirofumi; Yokota, Kunihiro; Kameda, Tatsuya; Yamagishi, Toshio
2010-01-01
Stress hormones have been associated with temporal discounting. Although time-discount rate is shown to be stable over a long term, no study to date examines whether individual differences in stress hormones could predict individuals' time-discount rates in the relatively distant future (e.g., six month later), which is of interest in neuroeconomics of stress-addiction association. We assessed 87 participants' salivary stress hormone (cortisol, cortisone, and alpha-amylase) levels and hyperbolic discounting of delayed rewards consisting of three magnitudes, at the time-interval of six months. For salivary steroid assays, we employed a liquid chromatography/ mass spectroscopy (LC/MS) method. The correlations between the stress hormone levels and time-discount rates were examined. We observed that salivary alpha-amylase (sAA) levels were negatively associated with time-discount rates in never-smokers. Notably, salivary levels of stress steroids (i.e., cortisol and cortisone) negatively and positively related to time-discount rates in men and women, respectively, in never-smokers. Ever-smokers' discount rates were not predicted from these stress hormone levels. Individual differences in stress hormone levels predict impulsivity in temporal discounting in the future. There are sex differences in the effect of stress steroids on temporal discounting; while there was no sex defference in the relationship between sAA and temporal discounting.
-
Peterson, Jennifer R.; Hill, Catherine C.; Kirkpatrick, Kimberly
2016-01-01
Impulsive choice is typically measured by presenting smaller-sooner (SS) versus larger-later (LL) rewards, with biases towards the SS indicating impulsivity. The current study tested rats on different impulsive choice procedures with LL delay manipulations to assess same-form and alternate-form test-retest reliability. In the systematic-GE procedure (Green & Estle, 2003), the LL delay increased after several sessions of training; in the systematic-ER procedure (Evenden & Ryan, 1996), the delay increased within each session; and in the adjusting-M procedure (Mazur, 1987), the delay changed after each block of trials within a session based on each rat’s choices in the previous block. In addition to measuring choice behavior, we also assessed temporal tracking of the LL delays using the median times of responding during LL trials. The two systematic procedures yielded similar results in both choice and temporal tracking measures following extensive training, whereas the adjusting procedure resulted in relatively more impulsive choices and poorer temporal tracking. Overall, the three procedures produced acceptable same form test-retest reliability over time, but the adjusting procedure did not show significant alternate form test-retest reliability with the other two procedures. The results suggest that systematic procedures may supply better measurements of impulsive choice in rats. PMID:25490901
-
Attention Deficit Hyperactivity Disorder and Tuberous Sclerosis Complex
... the parents on establishing structure in the child’s environment, using effective discipline for impulsive behaviors, rewarding attention, reducing oppositional behavior, promoting positive parent- ...
-
[Psychometric properties of the French version of the Effort-Reward Imbalance model].
Niedhammer, I; Siegrist, J; Landre, M F; Goldberg, M; Leclerc, A
2000-10-01
Two main models are currently used to evaluate psychosocial factors at work: the Job Strain model developed by Karasek and the Effort-Reward Imbalance model. A French version of the first model has been validated for the dimensions of psychological demands and decision latitude. As regards the second one evaluating three dimensions (extrinsic effort, reward, and intrinsic effort), there are several versions in different languages, but until recently there was no validated French version. The objective of this study was to explore the psychometric properties of the French version of the Effort-Reward Imbalance model in terms of internal consistency, factorial validity, and discriminant validity. The present study was based on the GAZEL cohort and included the 10 174 subjects who were working at the French national electric and gas company (EDF-GDF) and answered the questionnaire in 1998. A French version of Effort-Reward Imbalance was included in this questionnaire. This version was obtained by a standard forward/backward translation procedure. Internal consistency was satisfactory for the three scales of extrinsic effort, reward, and intrinsic effort: Cronbach's Alpha coefficients higher than 0.7 were observed. A one-factor solution was retained for the factor analysis of the scale of extrinsic effort. A three-factor solution was retained for the factor analysis of reward, and these dimensions were interpreted as the factor analysis of intrinsic effort did not support the expected four-dimension structure. The analysis of discriminant validity displayed significant associations between measures of Effort-Reward Imbalance and the variables of sex, age, education level, and occupational grade. This study is the first one supporting satisfactory psychometric properties of the French version of the Effort-Reward Imbalance model. However, the factorial validity of intrinsic effort could be questioned. Furthermore, as most previous studies were based on male samples working in specific occupations, the present one is also one of the first to show strong associations between measures of this model and social class variables in a population of men and women employed in various occupations.
-
ERIC Educational Resources Information Center
Hirai, Masahiro; Hiraki, Kazuo
2006-01-01
We investigated how the spatiotemporal structure of animations of biological motion (BM) affects brain activity. We measured event-related potentials (ERPs) during the perception of BM under four conditions: normal spatial and temporal structure; scrambled spatial and normal temporal structure; normal spatial and scrambled temporal structure; and…
-
Frank, G K W; Shott, M E; Riederer, J; Pryor, T L
2016-01-01
Anorexia and bulimia nervosa are severe eating disorders that share many behaviors. Structural and functional brain circuits could provide biological links that those disorders have in common. We recruited 77 young adult women, 26 healthy controls, 26 women with anorexia and 25 women with bulimia nervosa. Probabilistic tractography was used to map white matter connectivity strength across taste and food intake regulating brain circuits. An independent multisample greedy equivalence search algorithm tested effective connectivity between those regions during sucrose tasting. Anorexia and bulimia nervosa had greater structural connectivity in pathways between insula, orbitofrontal cortex and ventral striatum, but lower connectivity from orbitofrontal cortex and amygdala to the hypothalamus (P<0.05, corrected for comorbidity, medication and multiple comparisons). Functionally, in controls the hypothalamus drove ventral striatal activity, but in anorexia and bulimia nervosa effective connectivity was directed from anterior cingulate via ventral striatum to the hypothalamus. Across all groups, sweetness perception was predicted by connectivity strength in pathways connecting to the middle orbitofrontal cortex. This study provides evidence that white matter structural as well as effective connectivity within the energy-homeostasis and food reward-regulating circuitry is fundamentally different in anorexia and bulimia nervosa compared with that in controls. In eating disorders, anterior cingulate cognitive–emotional top down control could affect food reward and eating drive, override hypothalamic inputs to the ventral striatum and enable prolonged food restriction. PMID:27801897
-
Jankowski, Kathryn F; Takahashi, Hidehiko
2014-05-01
Social emotions are affective states elicited during social interactions and integral for promoting socially appropriate behaviors and discouraging socially inappropriate ones. Social emotion-processing deficits significantly impair interpersonal relationships, and play distinct roles in the manifestation and maintenance of clinical symptomatology. Elucidating the neural correlates of discrete social emotions can serve as a window to better understanding and treating neuropsychiatric disorders. Moral cognition and social emotion-processing broadly recruit a fronto-temporo-subcortical network, supporting empathy, perspective-taking, self-processing, and reward-processing. The present review specifically examines the neural correlates of embarrassment, guilt, envy, and schadenfreude. Embarrassment and guilt are self-conscious emotions, evoked during negative evaluation following norm violations and supported by a fronto-temporo-posterior network. Embarrassment is evoked by social transgressions and recruits greater anterior temporal regions, representing conceptual social knowledge. Guilt is evoked by moral transgressions and recruits greater prefrontal regions, representing perspective-taking and behavioral change demands. Envy and schadenfreude are fortune-of-other emotions, evoked during social comparison and supported by a prefronto-striatal network. Envy represents displeasure in others' fortunes, and recruits increased dorsal anterior cingulate cortex, representing cognitive dissonance, and decreased reward-related striatal regions. Schadenfreude represents pleasure in others' misfortunes, and recruits reduced empathy-related insular regions and increased reward-related striatal regions. Implications for psychopathology and treatment design are discussed. © 2014 The Authors. Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology.
-
Kishida, Kenneth T.; Saez, Ignacio; Lohrenz, Terry; Witcher, Mark R.; Laxton, Adrian W.; Tatter, Stephen B.; White, Jason P.; Ellis, Thomas L.; Phillips, Paul E. M.; Montague, P. Read
2016-01-01
In the mammalian brain, dopamine is a critical neuromodulator whose actions underlie learning, decision-making, and behavioral control. Degeneration of dopamine neurons causes Parkinson’s disease, whereas dysregulation of dopamine signaling is believed to contribute to psychiatric conditions such as schizophrenia, addiction, and depression. Experiments in animal models suggest the hypothesis that dopamine release in human striatum encodes reward prediction errors (RPEs) (the difference between actual and expected outcomes) during ongoing decision-making. Blood oxygen level-dependent (BOLD) imaging experiments in humans support the idea that RPEs are tracked in the striatum; however, BOLD measurements cannot be used to infer the action of any one specific neurotransmitter. We monitored dopamine levels with subsecond temporal resolution in humans (n = 17) with Parkinson’s disease while they executed a sequential decision-making task. Participants placed bets and experienced monetary gains or losses. Dopamine fluctuations in the striatum fail to encode RPEs, as anticipated by a large body of work in model organisms. Instead, subsecond dopamine fluctuations encode an integration of RPEs with counterfactual prediction errors, the latter defined by how much better or worse the experienced outcome could have been. How dopamine fluctuations combine the actual and counterfactual is unknown. One possibility is that this process is the normal behavior of reward processing dopamine neurons, which previously had not been tested by experiments in animal models. Alternatively, this superposition of error terms may result from an additional yet-to-be-identified subclass of dopamine neurons. PMID:26598677
-
Koch, Stefan P.; Hägele, Claudia; Haynes, John-Dylan; Heinz, Andreas; Schlagenhauf, Florian; Sterzer, Philipp
2015-01-01
Functional neuroimaging has provided evidence for altered function of mesolimbic circuits implicated in reward processing, first and foremost the ventral striatum, in patients with schizophrenia. While such findings based on significant group differences in brain activations can provide important insights into the pathomechanisms of mental disorders, the use of neuroimaging results from standard univariate statistical analysis for individual diagnosis has proven difficult. In this proof of concept study, we tested whether the predictive accuracy for the diagnostic classification of schizophrenia patients vs. healthy controls could be improved using multivariate pattern analysis (MVPA) of regional functional magnetic resonance imaging (fMRI) activation patterns for the anticipation of monetary reward. With a searchlight MVPA approach using support vector machine classification, we found that the diagnostic category could be predicted from local activation patterns in frontal, temporal, occipital and midbrain regions, with a maximal cluster peak classification accuracy of 93% for the right pallidum. Region-of-interest based MVPA for the ventral striatum achieved a maximal cluster peak accuracy of 88%, whereas the classification accuracy on the basis of standard univariate analysis reached only 75%. Moreover, using support vector regression we could additionally predict the severity of negative symptoms from ventral striatal activation patterns. These results show that MVPA can be used to substantially increase the accuracy of diagnostic classification on the basis of task-related fMRI signal patterns in a regionally specific way. PMID:25799236
-
Kishida, Kenneth T; Saez, Ignacio; Lohrenz, Terry; Witcher, Mark R; Laxton, Adrian W; Tatter, Stephen B; White, Jason P; Ellis, Thomas L; Phillips, Paul E M; Montague, P Read
2016-01-05
In the mammalian brain, dopamine is a critical neuromodulator whose actions underlie learning, decision-making, and behavioral control. Degeneration of dopamine neurons causes Parkinson's disease, whereas dysregulation of dopamine signaling is believed to contribute to psychiatric conditions such as schizophrenia, addiction, and depression. Experiments in animal models suggest the hypothesis that dopamine release in human striatum encodes reward prediction errors (RPEs) (the difference between actual and expected outcomes) during ongoing decision-making. Blood oxygen level-dependent (BOLD) imaging experiments in humans support the idea that RPEs are tracked in the striatum; however, BOLD measurements cannot be used to infer the action of any one specific neurotransmitter. We monitored dopamine levels with subsecond temporal resolution in humans (n = 17) with Parkinson's disease while they executed a sequential decision-making task. Participants placed bets and experienced monetary gains or losses. Dopamine fluctuations in the striatum fail to encode RPEs, as anticipated by a large body of work in model organisms. Instead, subsecond dopamine fluctuations encode an integration of RPEs with counterfactual prediction errors, the latter defined by how much better or worse the experienced outcome could have been. How dopamine fluctuations combine the actual and counterfactual is unknown. One possibility is that this process is the normal behavior of reward processing dopamine neurons, which previously had not been tested by experiments in animal models. Alternatively, this superposition of error terms may result from an additional yet-to-be-identified subclass of dopamine neurons.
-
Wang, Shuai; Shi, Yi; Li, Bao-Ming
2017-03-01
The anterior cingulate cortex (ACC) is crucial for decision making which involves the processing of cost-benefit information. Our previous study has shown that ACC is essential for self-paced decision making. However, it is unclear how ACC neurons represent cost-benefit selections during the decision-making process. In the present study, we trained rats on the same "Do More Get More" (DMGM) task as in our previous work. In each trial, the animals stand upright and perform a sustained nosepoke of their own will to earn a water reward, with the amount of reward positively correlated to the duration of the nosepoke (i.e., longer nosepokes earn larger rewards). We then recorded ACC neuronal activity on well-trained rats while they were performing the DMGM task. Our results show that (1) approximately 3/5 ACC neurons (296/496, 59.7%) exhibited changes in firing frequency that were temporally locked with the main events of the DMGM task; (2) about 1/5 ACC neurons (101/496, 20.4%) or 1/3 of the event-modulated neurons (101/296, 34.1%) showed differential firing rate changes for different cost-benefit selections; and (3) many ACC neurons exhibited linear encoding of the cost-benefit selections in the DMGM task events. These results suggest that ACC neurons are engaged in encoding cost-benefit information, thus represent the selections in self-paced decision making. Copyright © 2016 Elsevier Inc. All rights reserved.
-
Donaldson, Kayla R; Ait Oumeziane, Belel; Hélie, Sebastien; Foti, Dan
2016-07-01
Adapting behavior to dynamic stimulus-reward contingences is a core feature of reversal learning and a capacity thought to be critical to socio-emotional behavior. Impairment in reversal learning has been linked to multiple psychiatric outcomes, including depression, Parkinson's disorder, and substance abuse. A recent influential study introduced an innovative laboratory reversal-learning paradigm capable of disentangling the roles of feedback valence and expectancy. Here, we sought to use this paradigm in order to examine the time-course of reward and punishment learning using event-related potentials among a large, representative sample (N=101). Three distinct phases of processing were examined: initial feedback evaluation (reward positivity, or RewP), allocation of attention (P3), and sustained processing (late positive potential, or LPP). Results indicate a differential pattern of valence and expectancy across these processing stages: the RewP was uniquely related to valence (i.e., positive vs. negative feedback), the P3 was uniquely associated with expectancy (i.e., unexpected vs. expected feedback), and the LPP was sensitive to both valence and expectancy (i.e., main effects of each, but no interaction). The link between ERP amplitudes and behavioral performance was strongest for the P3, and this association was valence-specific. Overall, these findings highlight the potential utility of the P3 as a neural marker for feedback processing in reversal-based learning and establish a foundation for future research in clinical populations. Copyright © 2016 Elsevier Inc. All rights reserved.
-
Beyond discounting: possible experimental models of impulse control.
Monterosso, J; Ainslie, G
1999-10-01
Animal studies of impulsivity have typically used one of three models: a delay of reward procedure, a differential reinforcement for low rate responding (DRL) procedure, or an autoshaping procedure. In each of these paradigms, we argue, measurement of impulsivity is implicitly or explicitly equated with the effect delay has on the value of reward. The steepness by which delay diminishes value (the temporal discount function) is treated as an index of impulsivity. In order to provide a better analog of human impulsivity, this model needs to be expanded to include the converse of impulsivity - self-control. Through mechanisms such as committing to long range interests before the onset of temptation, or through bundling individual choices into classes of choices that are made at once, human decision-making can often look far less myopic than single trial experiments predict. For people, impulsive behavior may be more often the result of the breakdown of self-control mechanisms than of steep discount functions. Existing animal models of self-control are discussed, and future directions are suggested for psychopharmacological research.
-
Derivatives of logarithmic stationary distributions for policy gradient reinforcement learning.
Morimura, Tetsuro; Uchibe, Eiji; Yoshimoto, Junichiro; Peters, Jan; Doya, Kenji
2010-02-01
Most conventional policy gradient reinforcement learning (PGRL) algorithms neglect (or do not explicitly make use of) a term in the average reward gradient with respect to the policy parameter. That term involves the derivative of the stationary state distribution that corresponds to the sensitivity of its distribution to changes in the policy parameter. Although the bias introduced by this omission can be reduced by setting the forgetting rate gamma for the value functions close to 1, these algorithms do not permit gamma to be set exactly at gamma = 1. In this article, we propose a method for estimating the log stationary state distribution derivative (LSD) as a useful form of the derivative of the stationary state distribution through backward Markov chain formulation and a temporal difference learning framework. A new policy gradient (PG) framework with an LSD is also proposed, in which the average reward gradient can be estimated by setting gamma = 0, so it becomes unnecessary to learn the value functions. We also test the performance of the proposed algorithms using simple benchmark tasks and show that these can improve the performances of existing PG methods.
-
Griskevicius, Vladas; Tybur, Joshua M; Delton, Andrew W; Robertson, Theresa E
2011-06-01
Why do some people take risks and live for the present, whereas others avoid risks and save for the future? The evolutionary framework of life history theory predicts that preferences for risk and delay in gratification should be influenced by mortality and resource scarcity. A series of experiments examined how mortality cues influenced decisions involving risk preference (e.g., $10 for sure vs. 50% chance of $20) and temporal discounting (e.g., $5 now vs. $10 later). The effect of mortality depended critically on whether people grew up in a relatively resource-scarce or resource-plentiful environment. For individuals who grew up relatively poor, mortality cues led them to value the present and gamble for big immediate rewards. Conversely, for individuals who grew up relatively wealthy, mortality cues led them to value the future and avoid risky gambles. Overall, mortality cues appear to propel individuals toward diverging life history strategies as a function of childhood socioeconomic status, suggesting important implications for how environmental factors influence economic decisions and risky behaviors. 2011 APA, all rights reserved
-
Encoding of marginal utility across time in the human brain.
Pine, Alex; Seymour, Ben; Roiser, Jonathan P; Bossaerts, Peter; Friston, Karl J; Curran, H Valerie; Dolan, Raymond J
2009-07-29
Marginal utility theory prescribes the relationship between the objective property of the magnitude of rewards and their subjective value. Despite its pervasive influence, however, there is remarkably little direct empirical evidence for such a theory of value, let alone of its neurobiological basis. We show that human preferences in an intertemporal choice task are best described by a model that integrates marginally diminishing utility with temporal discounting. Using functional magnetic resonance imaging, we show that activity in the dorsal striatum encodes both the marginal utility of rewards, over and above that which can be described by their magnitude alone, and the discounting associated with increasing time. In addition, our data show that dorsal striatum may be involved in integrating subjective valuation systems inherent to time and magnitude, thereby providing an overall metric of value used to guide choice behavior. Furthermore, during choice, we show that anterior cingulate activity correlates with the degree of difficulty associated with dissonance between value and time. Our data support an integrative architecture for decision making, revealing the neural representation of distinct subcomponents of value that may contribute to impulsivity and decisiveness.
-
A Quantitative Relationship between Signal Detection in Attention and Approach/Avoidance Behavior
Viswanathan, Vijay; Sheppard, John P.; Kim, Byoung W.; Plantz, Christopher L.; Ying, Hao; Lee, Myung J.; Raman, Kalyan; Mulhern, Frank J.; Block, Martin P.; Calder, Bobby; Lee, Sang; Mortensen, Dale T.; Blood, Anne J.; Breiter, Hans C.
2017-01-01
This study examines how the domains of reward and attention, which are often studied as independent processes, in fact interact at a systems level. We operationalize divided attention with a continuous performance task and variables from signal detection theory (SDT), and reward/aversion with a keypress task measuring approach/avoidance in the framework of relative preference theory (RPT). Independent experiments with the same subjects showed a significant association between one SDT and two RPT variables, visualized as a three-dimensional structure. Holding one of these three variables constant, further showed a significant relationship between a loss aversion-like metric from the approach/avoidance task, and the response bias observed during the divided attention task. These results indicate that a more liberal response bias under signal detection (i.e., a higher tolerance for noise, resulting in a greater proportion of false alarms) is associated with higher “loss aversion.” Furthermore, our functional model suggests a mechanism for processing constraints with divided attention and reward/aversion. Together, our results argue for a systematic relationship between divided attention and reward/aversion processing in humans. PMID:28270776