Decision-Washington最新文献

英文中文

Cognitive Graphs: Representational Substrates for Planning. 认知图：规划的表征基础。

IF 1.2 Q3 PSYCHOLOGY, MULTIDISCIPLINARY

Decision-Washington

Pub Date : 2024-10-01 Epub Date: 2024-08-29 DOI: 10.1037/dec0000249

Jungsun Yoo, Elizabeth R Chrastil, Aaron M Bornstein

Making plans for upcoming actions is a computationally demanding process. To mitigate these demands, individuals can build extensive internal models of their environment-states, actions, and their sequential relationships-that allow for plans to be developed with minimal computational costs. Initially, these models reflect elaborate networks of learned associative relationships, which can be used to generate plans for reward through more iterative computations such as trajectory sampling. After sufficient experience, compressed forms of these models can efficiently capture long-range sequential structure, allowing them to be used for rapid planning even in pursuit of novel or changing rewards. Here, we review recent work on the multitude of representations that can support different forms of planning. We discuss how cognitive graphs, a framework with roots in both cognitive psychology and computer science, can provide a unifying view of these representations and their relationships to one another. Conceptualizing internal models as forms of graphs situates them on a spectrum where different kinds of structured sequences can be queried to support both planning and the formation of iteratively more compressed predictive representations. We discuss how each of these kinds of cognitive graphs are created during learning, and used to transfer and generalize knowledge across environments. Taken together, this review highlights the significant impact that the various associative structures of memory have on planning.

为即将到来的行动制定计划是一个计算要求很高的过程。为了减轻这些需求，个人可以建立广泛的内部模型，包括他们的环境状态、行动及其顺序关系，从而允许以最小的计算成本制定计划。最初，这些模型反映了学习联想关系的复杂网络，可以通过更多的迭代计算（如轨迹采样）来生成奖励计划。经过充分的经验，这些模型的压缩形式可以有效地捕获长期顺序结构，允许它们用于快速规划，甚至在追求新颖或变化的奖励时。在这里，我们回顾了最近关于支持不同形式规划的众多表示的工作。我们将讨论认知图（一个根植于认知心理学和计算机科学的框架）如何为这些表征及其相互关系提供统一的观点。将内部模型概念化为图形的形式，将它们置于可以查询不同类型的结构化序列的频谱上，以支持规划和迭代更压缩的预测表示的形成。我们讨论了这些类型的认知图是如何在学习过程中创建的，并用于跨环境转移和概括知识。总之，这篇综述强调了记忆的各种联想结构对计划的重要影响。

{"title":"Cognitive Graphs: Representational Substrates for Planning.","authors":"Jungsun Yoo, Elizabeth R Chrastil, Aaron M Bornstein","doi":"10.1037/dec0000249","DOIUrl":"10.1037/dec0000249","url":null,"abstract":"Making plans for upcoming actions is a computationally demanding process. To mitigate these demands, individuals can build extensive internal models of their environment-states, actions, and their sequential relationships-that allow for plans to be developed with minimal computational costs. Initially, these models reflect elaborate networks of learned associative relationships, which can be used to generate plans for reward through more iterative computations such as trajectory sampling. After sufficient experience, compressed forms of these models can efficiently capture long-range sequential structure, allowing them to be used for rapid planning even in pursuit of novel or changing rewards. Here, we review recent work on the multitude of representations that can support different forms of planning. We discuss how cognitive graphs, a framework with roots in both cognitive psychology and computer science, can provide a unifying view of these representations and their relationships to one another. Conceptualizing internal models as forms of graphs situates them on a spectrum where different kinds of structured sequences can be queried to support both planning and the formation of iteratively more compressed predictive representations. We discuss how each of these kinds of cognitive graphs are created during learning, and used to transfer and generalize knowledge across environments. Taken together, this review highlights the significant impact that the various associative structures of memory have on planning.","PeriodicalId":29935,"journal":{"name":"Decision-Washington","volume":"11 4","pages":"537-556"},"PeriodicalIF":1.2,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12520196/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145303756","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Discounting Future Reward in an Uncertain World. 在一个不确定的世界里贴现未来的回报。

IF 1 Q3 PSYCHOLOGY, MULTIDISCIPLINARY

Decision-Washington

Pub Date : 2024-04-01 Epub Date: 2023-06-29 DOI: 10.1037/dec0000219

G W Story, Z Kurth-Nelson, M Moutoussis, K Iigaya, G-J Will, T U Hauser, B Blain, I Vlaev, R J Dolan

Humans discount delayed relative to more immediate reward. A plausible explanation is that impatience arises partly from uncertainty, or risk, implicit in delayed reward. Existing theories of discounting-as-risk focus on a probability that delayed reward will not materialize. By contrast, we examine how uncertainty in the magnitude of delayed reward contributes to delay discounting. We propose a model wherein reward is discounted proportional to the rate of random change in its magnitude across time, termed volatility. We find evidence to support this model across three experiments (total N = 158). First, using a task where participants chose when to sell products, whose price dynamics they previously learned, we show discounting increases in line with price volatility. Second, we show that this effect pertains over naturalistic delays of up to 4 months. Using functional magnetic resonance imaging, we observe a volatility-dependent decrease in functional hippocampal-prefrontal coupling during intertemporal choice. Third, we replicate these effects in a larger online sample, finding that volatility discounting within each task correlates with baseline discounting outside of the task. We conclude that delay discounting partly reflects time-dependent uncertainty about reward magnitude, that is volatility. Our model captures how discounting adapts to volatility, thereby partly accounting for individual differences in impatience. Our imaging findings suggest a putative mechanism whereby uncertainty reduces prospective simulation of future outcomes.

相对于更直接的奖励，人类更看重延迟。一种合理的解释是，不耐烦部分源于不确定性或风险，这些不确定性或风险隐含在延迟的回报中。现有的风险贴现理论关注的是延迟回报不会实现的概率。相比之下，我们研究了延迟奖励大小的不确定性如何影响延迟折扣。我们提出了一个模型，其中奖励与随时间的随机变化率成比例，称为波动率。我们在三个实验（总N = 158）中找到了支持该模型的证据。首先，使用一个任务，参与者选择何时出售产品，他们之前已经了解了产品的价格动态，我们显示折扣的增加与价格波动一致。其次，我们表明这种效应适用于长达4个月的自然延迟。使用功能性磁共振成像，我们观察到在颞间选择过程中海马-前额叶功能耦合的波动依赖性下降。第三，我们在一个更大的在线样本中复制了这些效应，发现每个任务内的波动率折扣与任务外的基线折扣相关。我们得出结论，延迟折扣部分反映了报酬大小的时间依赖性不确定性，即波动性。我们的模型捕捉到了贴现是如何适应波动性的，从而在一定程度上解释了不耐烦程度的个体差异。我们的影像学发现提示了一种假定的机制，即不确定性降低了对未来结果的预期模拟。

{"title":"Discounting Future Reward in an Uncertain World.","authors":"G W Story, Z Kurth-Nelson, M Moutoussis, K Iigaya, G-J Will, T U Hauser, B Blain, I Vlaev, R J Dolan","doi":"10.1037/dec0000219","DOIUrl":"10.1037/dec0000219","url":null,"abstract":"Humans discount delayed relative to more immediate reward. A plausible explanation is that impatience arises partly from uncertainty, or risk, implicit in delayed reward. Existing theories of discounting-as-risk focus on a probability that delayed reward will not materialize. By contrast, we examine how uncertainty in the magnitude of delayed reward contributes to delay discounting. We propose a model wherein reward is discounted proportional to the rate of random change in its magnitude across time, termed volatility. We find evidence to support this model across three experiments (total N = 158). First, using a task where participants chose when to sell products, whose price dynamics they previously learned, we show discounting increases in line with price volatility. Second, we show that this effect pertains over naturalistic delays of up to 4 months. Using functional magnetic resonance imaging, we observe a volatility-dependent decrease in functional hippocampal-prefrontal coupling during intertemporal choice. Third, we replicate these effects in a larger online sample, finding that volatility discounting within each task correlates with baseline discounting outside of the task. We conclude that delay discounting partly reflects time-dependent uncertainty about reward magnitude, that is volatility. Our model captures how discounting adapts to volatility, thereby partly accounting for individual differences in impatience. Our imaging findings suggest a putative mechanism whereby uncertainty reduces prospective simulation of future outcomes.","PeriodicalId":29935,"journal":{"name":"Decision-Washington","volume":"1 1","pages":"255-282"},"PeriodicalIF":1.0,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11949085/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41740821","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Correction to Lee and Holyoak (2023). 更正李和霍利亚克（2023）。