Suri, Roland
Coauthors(s): Bargas J, Departamento de Biofisica, Universidad Nacional Autonoma de Mexico, Mexico City Arbib M.A., USC Brain Project, Los Angeles
Salk Institute
CNL
P.O. Box 58800, San Diego, CA, 92186-5800
www.cnl.salk.edu/~suri

Modeling Functions of Striatal Dopamine Modulation in Learning and Planning

The activity of midbrain dopamine neurons is strikingly similar to the reward prediction error of TD reinforcement learning models. Experimental evidence and simulation studies suggest that dopamine neuron activity serves as an effective reinforcement signal for learning of sensorimotor associations in striatal matrisomes. In the current study, we simulate dopamine neuron activity with the Extended TD model (Suri and Schultz, submitted) and examine the influence of this signal on medium spiny neurons in striatal matrisomes. This model includes transient membrane effects of dopamine, dopamine-dependent long-term adaptations of corticostriatal transmission, and rhythmic fluctuations of the membrane potential between an elevated 'up-state' and a hyperpolarized 'down-state'? The most dominant activity in the striatal matrisomes elicits behaviors via projections from the basal ganglia to the thalamus and the cortex. This 'standard model' performs successfully when tested for sensorimotor learning and goal-directed behavior (planning). To investigate the contributions of these model assumptions to learning and planning, we test the performance of several model variants that lack one of these mechanisms. These simulations show that the adaptation of the dopamine-like signal is necessary for planning and for sensorimotor learning. Lack of dopamine-like novelty responses decreases the number of exploratory acts, which deteriorates planning capabilities. Sensorimotor learning requires dopamine-dependent long-term adaptation of corticostriatal transmission. The model loses its planning capabilities if the dopamine-like signal is simulated with the original TD model. The capability for planning is improved by transient dopamine membrane effects, dopamine-dependent long-term effects on corticostriatal transmission, dopamine- and input-dependent influences on the durations of membrane potential fluctuations, and manipulations that prolong the reaction time of the model. These simulation results suggest that striatal dopamine is important for sensorimotor learning, exploration, and planning. (Supported by the Swiss NSF (R.S.) and a program project grant from the Human Brain Project (M.A.); code and submitted paper at http://www.cnl.salk.edu/~suri/RES_bec.html)