气候变化领域集成服务门户(CCPortal): Human subjects exploit a cognitive map for credit assignment

CCPortal

DOI	10.1073/pnas.2016884118
	Human subjects exploit a cognitive map for credit assignment
	Moran R.; Dayan P.; Dolan R.J.
发表日期	2021
ISSN	00278424
卷号	118 期号:4
英文摘要	An influential reinforcement learning framework proposes that behavior is jointly governed by model-free (MF) and model-based (MB) controllers. The former learns the values of actions directly from past encounters, and the latter exploits a cognitive map of the task to calculate these prospectively. Considerable attention has been paid to how these systems interact during choice, but how and whether knowledge of a cognitive map contributes to the way MF and MB controllers assign credit (i.e., to how they revaluate actions and states following the receipt of an outcome) remains underexplored. Here, we examine such sophisticated credit assignment using a dual-outcome bandit task. We provide evidence that knowledge of a cognitive map influences credit assignment in both MF and MB systems, mediating subtly different aspects of apparent relevance. Specifically, we show MF credit assignment is enhanced for those rewards that are related to a choice, and this contrasted with choice-unrelated rewards that reinforced subsequent choices negatively. This modulation is only possible based on knowledge of task structure. On the other hand, MB credit assignment was boosted for outcomes that impacted on differences in values between offered bandits. We consider mechanistic accounts and the normative status of these findings. We suggest the findings extend the scope and sophistication of cognitive map-based credit assignment during reinforcement learning, with implications for understanding behavioral control. © 2021 National Academy of Sciences. All rights reserved.
英文关键词	Cognitive maps; Decision making; Model-based; Model-free; Reinforcement learning
语种	英语
scopus关键词	adult; article; cognitive map; decision making; human; reinforcement; reward
来源期刊	Proceedings of the National Academy of Sciences of the United States of America
文献类型	期刊论文
条目标识符	http://gcip.llas.ac.cn/handle/2XKMVOVA/180885
作者单位	Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, WC1B 5EH, United Kingdom; Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3BG, United Kingdom; Department of Computational Neuroscience, Max Planck Institute for Biological Cybernetics, Tübingen, 72076, Germany; Department of Computer Science, University of Tübingen, Tübingen, 72076, Germany
推荐引用方式 GB/T 7714	Moran R.,Dayan P.,Dolan R.J.. Human subjects exploit a cognitive map for credit assignment[J],2021,118(4).
APA	Moran R.,Dayan P.,&Dolan R.J..(2021).Human subjects exploit a cognitive map for credit assignment.Proceedings of the National Academy of Sciences of the United States of America,118(4).
MLA	Moran R.,et al."Human subjects exploit a cognitive map for credit assignment".Proceedings of the National Academy of Sciences of the United States of America 118.4(2021).