Learning, contingency and credit assignment
How do we learn causal connections between events that occur in our world and our actions?
In many situations outside of the laboratory, we are faced with numerous options to decide between and the outcomes associated with each may be uncertain, delayed and changeable. This means it can be necessary to integrate across the history of recent past occurrences to build up a representation of the likely consequences of a choice. Similarly, it can be important to distinguish between associations between environmental stimuli or actions and outcomes, and, in the case of actions, whether these were self-generated or externally instructed. This is made even more taxing in such complex environments by the fact that it is frequently not straightforward to know what caused a particular outcome to occur.
We have found that frontal cortical regions are critical for the appropriate integration of recent reinforcement to guide beneficial decision making. Anterior cingulate cortex is required to integrate across the recent history of outcomes associated with particular actions to determine what response to make and how quickly to update value representations upon receipt of new information. We have recently shown that central and lateral parts of orbitofrontal cortex are critical for the appropriate credit assignment which can allow rapid stimulus-outcome learning in complex, changeable environments. Such a system is separate to another slower learning system that approximates associations based on the recent history of chosen stimuli and received outcomes.
Current research focuses on:
- The role of orbitofrontal cortex (OFC) in credit assignment
- Electrochemical studies of associative learning in complex environments
Selected Key Papers
Reviews:
- Walton ME, Gan JO, Phillips PE (in press). The influence of dopamine in generating action from motivation. In RB Mars, J Sallet, MF Rushworth, & N Young (Eds.) Neural basis of Motivation and Cognitive Control. Cambridge: MIT press.
- Walton ME, Croxson PL, Behrens TJ, Kennerley SW, Rushworth MFS (2007). Adaptive decision making and value in the anterior cingulate cortex. Neuroimage 36, T142-54.
Primary research papers:
- Walton ME*, Behrens TE*, Rudebeck PH, Baxter MG, Buckley MJ, Rushworth MF (2010). Separable learning systems in the macaque brain and the role of the orbitofrontal cortex in contingent learning. Neuron 65: 927-39. (*, these authors contributed equally to this work)
- Kennerley SW, Walton ME, Behrens TE, Buckley MJ, Rushworth MFS (2006). Optimal decision making and the anterior cingulate cortex. Nat Neurosci. 9: 940-947.
- Walton ME, Devlin JT, Rushworth MFS (2004). Interactions between decision making and feedback monitoring within prefrontal cortex. Nat Neurosci. 7: 1259-1265.