Pub Date : 2013-03-01DOI: 10.1109/TAMD.2012.2211101
Chunguang Li, Yuke Li
The discovery of neuronal intrinsic plasticity (IP) processes which persistently modify a neuron's excitability necessitates a new concept of the neuronal plasticity mechanism and may profoundly influence our ideas on learning and memory. In this paper, we propose a spike-based IP model/adaptation rule for an integrate-and-fire (IF) neuron to model this biological phenomenon. By utilizing spikes denoted by Dirac delta functions rather than computing instantaneous firing rates for the time-dependent stimulus, this simple adaptation rule adjusts two parameters of an individual IF neuron to modify its excitability. As a result, this adaptation rule helps an IF neuron to keep its firing activity in a relatively “low but not too low” level and makes the spike-count distributions computed with adjusted window sizes similar to the experimental results.
{"title":"A Spike-Based Model of Neuronal Intrinsic Plasticity","authors":"Chunguang Li, Yuke Li","doi":"10.1109/TAMD.2012.2211101","DOIUrl":"https://doi.org/10.1109/TAMD.2012.2211101","url":null,"abstract":"The discovery of neuronal intrinsic plasticity (IP) processes which persistently modify a neuron's excitability necessitates a new concept of the neuronal plasticity mechanism and may profoundly influence our ideas on learning and memory. In this paper, we propose a spike-based IP model/adaptation rule for an integrate-and-fire (IF) neuron to model this biological phenomenon. By utilizing spikes denoted by Dirac delta functions rather than computing instantaneous firing rates for the time-dependent stimulus, this simple adaptation rule adjusts two parameters of an individual IF neuron to modify its excitability. As a result, this adaptation rule helps an IF neuron to keep its firing activity in a relatively “low but not too low” level and makes the spike-count distributions computed with adjusted window sizes similar to the experimental results.","PeriodicalId":49193,"journal":{"name":"IEEE Transactions on Autonomous Mental Development","volume":"5 1","pages":"62-73"},"PeriodicalIF":0.0,"publicationDate":"2013-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAMD.2012.2211101","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62760743","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-12-01DOI: 10.1109/TAMD.2012.2199989
Jonas Ruesch, R. Ferreira, A. Bernardino
Neural circuits that route motor activity to sensory structures play a fundamental role in perception. Their purpose is to aid basic cognitive processes by integrating knowledge about an organism's actions and to predict the perceptual consequences of those actions. This work develops a biologically inspired model of a visual stimulus prediction circuit and proposes a mathematical formulation for a computational implementation. We consider an agent with a visual sensory area consisting of an unknown rigid configuration of light-sensitive receptive fields which move with respect to the environment and according to a given number of degrees of freedom. From the agent's perspective, every movement induces an initially unknown change to the recorded stimulus. In line with evidence collected from studies on ontogenetic development and the plasticity of neural circuits, the proposed model adapts its structure with respect to experienced stimuli collected during the execution of a set of exploratory actions. We discuss the tendency of the proposed model to organize such that the prediction function is built using a particularly sparse feedforward network which requires a minimum amount of wiring and computational operations. We also observe a dualism between the organization of an intermediate layer of the network and the concept of self-similarity.
{"title":"Predicting Visual Stimuli From Self-Induced Actions: An Adaptive Model of a Corollary Discharge Circuit","authors":"Jonas Ruesch, R. Ferreira, A. Bernardino","doi":"10.1109/TAMD.2012.2199989","DOIUrl":"https://doi.org/10.1109/TAMD.2012.2199989","url":null,"abstract":"Neural circuits that route motor activity to sensory structures play a fundamental role in perception. Their purpose is to aid basic cognitive processes by integrating knowledge about an organism's actions and to predict the perceptual consequences of those actions. This work develops a biologically inspired model of a visual stimulus prediction circuit and proposes a mathematical formulation for a computational implementation. We consider an agent with a visual sensory area consisting of an unknown rigid configuration of light-sensitive receptive fields which move with respect to the environment and according to a given number of degrees of freedom. From the agent's perspective, every movement induces an initially unknown change to the recorded stimulus. In line with evidence collected from studies on ontogenetic development and the plasticity of neural circuits, the proposed model adapts its structure with respect to experienced stimuli collected during the execution of a set of exploratory actions. We discuss the tendency of the proposed model to organize such that the prediction function is built using a particularly sparse feedforward network which requires a minimum amount of wiring and computational operations. We also observe a dualism between the organization of an intermediate layer of the network and the concept of self-similarity.","PeriodicalId":49193,"journal":{"name":"IEEE Transactions on Autonomous Mental Development","volume":"4 1","pages":"290-304"},"PeriodicalIF":0.0,"publicationDate":"2012-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAMD.2012.2199989","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62760831","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-12-01DOI: 10.1109/TAMD.2012.2182765
Jun Yin, Y. Meng, Yaochu Jin
Reservoir computing (RC) is a computational framework for neural network based information processing. Little work, however, has been conducted on adapting the structure of the neural reservoir. In this paper, we propose a developmental approach to structural self-organization in reservoir computing. More specifically, a recurrent spiking neural network is adopted for building up the reservoir, whose synaptic and structural plasticity are regulated by a gene regulatory network (GRN). Meanwhile, the expression dynamics of the GRN is directly influenced by the activity of the neurons in the reservoir. We term this proposed model as GRN-regulated self-organizing RC (GRN-SO-RC). Contrary to a randomly initialized and fixed structure used in most existing RC models, the structure of the reservoir in the GRN-SO-RC model is self-organized to adapt to the specific task using the GRN-based mechanism. To evaluate the proposed model, experiments have been conducted on several benchmark problems widely used in RC models, such as memory capacity and nonlinear auto-regressive moving average. In addition, we apply the GRN-SO-RC model to solving complex real-world problems, including speech recognition and human action recognition. Our experimental results on both the benchmark and real-world problems demonstrate that the GRN-SO-RC model is effective and robust in solving different types of problems.
{"title":"A Developmental Approach to Structural Self-Organization in Reservoir Computing","authors":"Jun Yin, Y. Meng, Yaochu Jin","doi":"10.1109/TAMD.2012.2182765","DOIUrl":"https://doi.org/10.1109/TAMD.2012.2182765","url":null,"abstract":"Reservoir computing (RC) is a computational framework for neural network based information processing. Little work, however, has been conducted on adapting the structure of the neural reservoir. In this paper, we propose a developmental approach to structural self-organization in reservoir computing. More specifically, a recurrent spiking neural network is adopted for building up the reservoir, whose synaptic and structural plasticity are regulated by a gene regulatory network (GRN). Meanwhile, the expression dynamics of the GRN is directly influenced by the activity of the neurons in the reservoir. We term this proposed model as GRN-regulated self-organizing RC (GRN-SO-RC). Contrary to a randomly initialized and fixed structure used in most existing RC models, the structure of the reservoir in the GRN-SO-RC model is self-organized to adapt to the specific task using the GRN-based mechanism. To evaluate the proposed model, experiments have been conducted on several benchmark problems widely used in RC models, such as memory capacity and nonlinear auto-regressive moving average. In addition, we apply the GRN-SO-RC model to solving complex real-world problems, including speech recognition and human action recognition. Our experimental results on both the benchmark and real-world problems demonstrate that the GRN-SO-RC model is effective and robust in solving different types of problems.","PeriodicalId":49193,"journal":{"name":"IEEE Transactions on Autonomous Mental Development","volume":"4 1","pages":"273-289"},"PeriodicalIF":0.0,"publicationDate":"2012-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAMD.2012.2182765","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62760709","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-12-01DOI: 10.1109/TAMD.2012.2208962
J. Cabibihan, W. So, S. Pramanik
For robots to be accommodated in human spaces and in daily human activities, robots should be able to understand messages from their human conversation partner. In the same light, humans must also understand the messages that are being communicated to them by robots, including nonverbal messages. We conducted a Web-based video study wherein participants interpreted the iconic gestures and emblems produced by an anthropomorphic robot. Out of the 15 robotic gestures presented, we found 6 that can be accurately recognized by the human observer. These were nodding, clapping, hugging, expressing anger, walking, and flying. We review these gestures for their meaning from literature on human and animal behavior. We conclude by discussing the possible implications of these gestures for the design of social robots that are able to have engaging interactions with humans.
{"title":"Human-Recognizable Robotic Gestures","authors":"J. Cabibihan, W. So, S. Pramanik","doi":"10.1109/TAMD.2012.2208962","DOIUrl":"https://doi.org/10.1109/TAMD.2012.2208962","url":null,"abstract":"For robots to be accommodated in human spaces and in daily human activities, robots should be able to understand messages from their human conversation partner. In the same light, humans must also understand the messages that are being communicated to them by robots, including nonverbal messages. We conducted a Web-based video study wherein participants interpreted the iconic gestures and emblems produced by an anthropomorphic robot. Out of the 15 robotic gestures presented, we found 6 that can be accurately recognized by the human observer. These were nodding, clapping, hugging, expressing anger, walking, and flying. We review these gestures for their meaning from literature on human and animal behavior. We conclude by discussing the possible implications of these gestures for the design of social robots that are able to have engaging interactions with humans.","PeriodicalId":49193,"journal":{"name":"IEEE Transactions on Autonomous Mental Development","volume":"28 1","pages":"305-314"},"PeriodicalIF":0.0,"publicationDate":"2012-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAMD.2012.2208962","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62761029","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-12-01DOI: 10.1109/TAMD.2012.2205924
F. Stulp, J. Buchli, Alice Ellmer, M. Mistry, Evangelos A. Theodorou, S. Schaal
For humans and robots, variable impedance control is an essential component for ensuring robust and safe physical interaction with the environment. Humans learn to adapt their impedance to specific tasks and environments; a capability which we continually develop and improve until we are well into our twenties. In this article, we reproduce functionally interesting aspects of learning impedance control in humans on a simulated robot platform. As demonstrated in numerous force field tasks, humans combine two strategies to adapt their impedance to perturbations, thereby minimizing position error and energy consumption: 1) if perturbations are unpredictable, subjects increase their impedance through cocontraction; and 2) if perturbations are predictable, subjects learn a feed-forward command to offset the perturbation. We show how a 7-DOF simulated robot demonstrates similar behavior with our model-free reinforcement learning algorithm PI2, by applying deterministic and stochastic force fields to the robot's end-effector. We show the qualitative similarity between the robot and human movements. Our results provide a biologically plausible approach to learning appropriate impedances purely from experience, without requiring a model of either body or environment dynamics. Not requiring models also facilitates autonomous development for robots, as prespecified models cannot be provided for each environment a robot might encounter.
{"title":"Model-Free Reinforcement Learning of Impedance Control in Stochastic Environments","authors":"F. Stulp, J. Buchli, Alice Ellmer, M. Mistry, Evangelos A. Theodorou, S. Schaal","doi":"10.1109/TAMD.2012.2205924","DOIUrl":"https://doi.org/10.1109/TAMD.2012.2205924","url":null,"abstract":"For humans and robots, variable impedance control is an essential component for ensuring robust and safe physical interaction with the environment. Humans learn to adapt their impedance to specific tasks and environments; a capability which we continually develop and improve until we are well into our twenties. In this article, we reproduce functionally interesting aspects of learning impedance control in humans on a simulated robot platform. As demonstrated in numerous force field tasks, humans combine two strategies to adapt their impedance to perturbations, thereby minimizing position error and energy consumption: 1) if perturbations are unpredictable, subjects increase their impedance through cocontraction; and 2) if perturbations are predictable, subjects learn a feed-forward command to offset the perturbation. We show how a 7-DOF simulated robot demonstrates similar behavior with our model-free reinforcement learning algorithm PI2, by applying deterministic and stochastic force fields to the robot's end-effector. We show the qualitative similarity between the robot and human movements. Our results provide a biologically plausible approach to learning appropriate impedances purely from experience, without requiring a model of either body or environment dynamics. Not requiring models also facilitates autonomous development for robots, as prespecified models cannot be provided for each environment a robot might encounter.","PeriodicalId":49193,"journal":{"name":"IEEE Transactions on Autonomous Mental Development","volume":"15 1","pages":"330-341"},"PeriodicalIF":0.0,"publicationDate":"2012-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAMD.2012.2205924","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62761113","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-12-01DOI: 10.1109/TAMD.2012.2208457
K. Merrick
Incorporating intrinsic motivation with reinforcement learning can permit agents to independently choose, which skills they will develop, or to change their focus of attention to learn different skills at different times. This implies an autonomous developmental process for skills in which a skill-acquisition goal is first identified, then a skill is learned to solve the goal. The learned skill may then be stored, reused, temporarily ignored or even permanently erased. This paper formalizes the developmental process for skills by proposing a goal-lifecycle using the option framework for motivated reinforcement learning agents. The paper shows how the goal-lifecycle can be used as a basis for designing motivational state-spaces that permit agents to reason introspectively and autonomously about when to learn skills to solve goals, when to activate skills, when to suspend activation of skills or when to delete skills. An algorithm is presented that simultaneously learns: 1) an introspective policy mapping motivational states to decisions that change the agent's motivational state, and 2) multiple option policies mapping sensed states and actions to achieve various domain-specific goals. Two variations of agents using this model are compared to motivated reinforcement learning agents without introspection for controlling non-player characters in a computer game scenario. Results show that agents using introspection can focus their attention on learning more complex skills than agents without introspection. In addition, they can learn these skills more effectively.
{"title":"Intrinsic Motivation and Introspection in Reinforcement Learning","authors":"K. Merrick","doi":"10.1109/TAMD.2012.2208457","DOIUrl":"https://doi.org/10.1109/TAMD.2012.2208457","url":null,"abstract":"Incorporating intrinsic motivation with reinforcement learning can permit agents to independently choose, which skills they will develop, or to change their focus of attention to learn different skills at different times. This implies an autonomous developmental process for skills in which a skill-acquisition goal is first identified, then a skill is learned to solve the goal. The learned skill may then be stored, reused, temporarily ignored or even permanently erased. This paper formalizes the developmental process for skills by proposing a goal-lifecycle using the option framework for motivated reinforcement learning agents. The paper shows how the goal-lifecycle can be used as a basis for designing motivational state-spaces that permit agents to reason introspectively and autonomously about when to learn skills to solve goals, when to activate skills, when to suspend activation of skills or when to delete skills. An algorithm is presented that simultaneously learns: 1) an introspective policy mapping motivational states to decisions that change the agent's motivational state, and 2) multiple option policies mapping sensed states and actions to achieve various domain-specific goals. Two variations of agents using this model are compared to motivated reinforcement learning agents without introspection for controlling non-player characters in a computer game scenario. Results show that agents using introspection can focus their attention on learning more complex skills than agents without introspection. In addition, they can learn these skills more effectively.","PeriodicalId":49193,"journal":{"name":"IEEE Transactions on Autonomous Mental Development","volume":"4 1","pages":"315-329"},"PeriodicalIF":0.0,"publicationDate":"2012-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAMD.2012.2208457","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62760796","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-12-01DOI: 10.1109/TAMD.2012.2208640
H. Jasso, J. Triesch, G. Deák, J. Lewis
Gaze following, the ability to redirect one's visual attention to look at what another person is seeing, is foundational for imitation, word learning, and theory-of-mind. Previous theories have suggested that the development of gaze following in human infants is the product of a basic gaze following mechanism, plus the gradual incorporation of several distinct new mechanisms that improve the skill, such as spatial inference, and the ability to use eye direction information as well as head direction. In this paper, we offer an alternative explanation based on a single learning mechanism. From a starting state with no knowledge of the implications of another organism's gaze direction, our model learns to follow gaze by being placed in a simulated environment where an adult caregiver looks around at objects. Our infant model matches the development of gaze following in human infants as measured in key experiments that we replicate and analyze in detail.
{"title":"A Unified Account of Gaze Following","authors":"H. Jasso, J. Triesch, G. Deák, J. Lewis","doi":"10.1109/TAMD.2012.2208640","DOIUrl":"https://doi.org/10.1109/TAMD.2012.2208640","url":null,"abstract":"Gaze following, the ability to redirect one's visual attention to look at what another person is seeing, is foundational for imitation, word learning, and theory-of-mind. Previous theories have suggested that the development of gaze following in human infants is the product of a basic gaze following mechanism, plus the gradual incorporation of several distinct new mechanisms that improve the skill, such as spatial inference, and the ability to use eye direction information as well as head direction. In this paper, we offer an alternative explanation based on a single learning mechanism. From a starting state with no knowledge of the implications of another organism's gaze direction, our model learns to follow gaze by being placed in a simulated environment where an adult caregiver looks around at objects. Our infant model matches the development of gaze following in human infants as measured in key experiments that we replicate and analyze in detail.","PeriodicalId":49193,"journal":{"name":"IEEE Transactions on Autonomous Mental Development","volume":"4 1","pages":"257-272"},"PeriodicalIF":0.0,"publicationDate":"2012-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAMD.2012.2208640","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62760903","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-10DOI: 10.1109/TAMD.2012.2211475
Zhengyou Zhang
{"title":"Editorial: Impact Factor and Outstanding Paper Awards","authors":"Zhengyou Zhang","doi":"10.1109/TAMD.2012.2211475","DOIUrl":"https://doi.org/10.1109/TAMD.2012.2211475","url":null,"abstract":"","PeriodicalId":49193,"journal":{"name":"IEEE Transactions on Autonomous Mental Development","volume":"68 1","pages":"189"},"PeriodicalIF":0.0,"publicationDate":"2012-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85801397","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-01DOI: 10.1109/TAMD.2012.2211871
Richard Kelley, A. Tavakkoli, Christopher King, A. Ambardekar, M. Nicolescu, M. Nicolescu
One of the foundations of social interaction among humans is the ability to correctly identify interactions and infer the intentions of others. To build robots that reliably function in the human social world, we must develop models that robots can use to mimic the intent recognition skills found in humans. We propose a framework that uses contextual information in the form of object affordances and object state to improve the performance of an underlying intent recognition system. This system represents objects and their affordances using a directed graph that is automatically extracted from a large corpus of natural language text. We validate our approach on a physical robot that classifies intentions in a number of scenarios.
{"title":"Context-Based Bayesian Intent Recognition","authors":"Richard Kelley, A. Tavakkoli, Christopher King, A. Ambardekar, M. Nicolescu, M. Nicolescu","doi":"10.1109/TAMD.2012.2211871","DOIUrl":"https://doi.org/10.1109/TAMD.2012.2211871","url":null,"abstract":"One of the foundations of social interaction among humans is the ability to correctly identify interactions and infer the intentions of others. To build robots that reliably function in the human social world, we must develop models that robots can use to mimic the intent recognition skills found in humans. We propose a framework that uses contextual information in the form of object affordances and object state to improve the performance of an underlying intent recognition system. This system represents objects and their affordances using a directed graph that is automatically extracted from a large corpus of natural language text. We validate our approach on a physical robot that classifies intentions in a number of scenarios.","PeriodicalId":49193,"journal":{"name":"IEEE Transactions on Autonomous Mental Development","volume":"4 1","pages":"215-225"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAMD.2012.2211871","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62760857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2012-09-01DOI: 10.1109/TAMD.2012.2202658
Derrik E. Asher, Andrew Zaldivar, B. Barton, A. Brewer, J. Krichmar
Game theory has been useful for understanding risk-taking and cooperative behavior. However, in studies of the neural basis of decision-making during games of conflict, subjects typically play against opponents with predetermined strategies. The present study introduces a neurobiologically plausible model of action selection and neuromodulation, which adapts to its opponent's strategy and environmental conditions. The model is based on the assumption that dopaminergic and serotonergic systems track expected rewards and costs, respectively. The model controlled both simulated and robotic agents playing Hawk-Dove and Chicken games against subjects. When playing against an aggressive version of the model, there was a significant shift in the subjects' strategy from Win-Stay-Lose-Shift to Tit-For-Tat. Subjects became retaliatory when confronted with agents that tended towards risky behavior. These results highlight the important interactions between subjects and agents utilizing adaptive behavior. Moreover, they reveal neuromodulatory mechanisms that give rise to cooperative and competitive behaviors.
{"title":"Reciprocity and Retaliation in Social Games With Adaptive Agents","authors":"Derrik E. Asher, Andrew Zaldivar, B. Barton, A. Brewer, J. Krichmar","doi":"10.1109/TAMD.2012.2202658","DOIUrl":"https://doi.org/10.1109/TAMD.2012.2202658","url":null,"abstract":"Game theory has been useful for understanding risk-taking and cooperative behavior. However, in studies of the neural basis of decision-making during games of conflict, subjects typically play against opponents with predetermined strategies. The present study introduces a neurobiologically plausible model of action selection and neuromodulation, which adapts to its opponent's strategy and environmental conditions. The model is based on the assumption that dopaminergic and serotonergic systems track expected rewards and costs, respectively. The model controlled both simulated and robotic agents playing Hawk-Dove and Chicken games against subjects. When playing against an aggressive version of the model, there was a significant shift in the subjects' strategy from Win-Stay-Lose-Shift to Tit-For-Tat. Subjects became retaliatory when confronted with agents that tended towards risky behavior. These results highlight the important interactions between subjects and agents utilizing adaptive behavior. Moreover, they reveal neuromodulatory mechanisms that give rise to cooperative and competitive behaviors.","PeriodicalId":49193,"journal":{"name":"IEEE Transactions on Autonomous Mental Development","volume":"4 1","pages":"226-238"},"PeriodicalIF":0.0,"publicationDate":"2012-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAMD.2012.2202658","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"62761003","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：