Pub Date : 2018-07-01DOI: 10.1109/AGENTS.2018.8460062
Yusuke Nakata, Yuki Kitazato, S. Arai
Ideal products offer proper usages to users intuitively, and a usage perceived by a user is called an affordance. We aim to identify product features that induce the affordance of a specific action. We propose a method that identifies those affordance features without the need of an expert's knowledge of a domain. Using a dataset of a product's image and an affordance perceived by the product's user, the proposed method identifies those affordance features. The proposed method consists of three steps. First, we train a convolutional neural network (CNN) to predict a product's affordance. Second, according to the analysis of a trained CNN, we enumerate candidates for affordance features. Third, we use three metrics to verify and evaluate the candidates for features. By taking an affordance of “sit” as an example, our experiment showed that the proposed method does successfully identify affordance features.
{"title":"Detection of Features Affording a Certain Action via Analysis of CNN","authors":"Yusuke Nakata, Yuki Kitazato, S. Arai","doi":"10.1109/AGENTS.2018.8460062","DOIUrl":"https://doi.org/10.1109/AGENTS.2018.8460062","url":null,"abstract":"Ideal products offer proper usages to users intuitively, and a usage perceived by a user is called an affordance. We aim to identify product features that induce the affordance of a specific action. We propose a method that identifies those affordance features without the need of an expert's knowledge of a domain. Using a dataset of a product's image and an affordance perceived by the product's user, the proposed method identifies those affordance features. The proposed method consists of three steps. First, we train a convolutional neural network (CNN) to predict a product's affordance. Second, according to the analysis of a trained CNN, we enumerate candidates for affordance features. Third, we use three metrics to verify and evaluate the candidates for features. By taking an affordance of “sit” as an example, our experiment showed that the proposed method does successfully identify affordance features.","PeriodicalId":248901,"journal":{"name":"2018 IEEE International Conference on Agents (ICA)","volume":"69 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123008407","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-07-01DOI: 10.1109/AGENTS.2018.8460053
Siyuan Liu, Qiong Wu, C. Miao
In e-commerce, recommendation is an essential feature to provide users with potentially interesting items to purchase. However, people are often faced with an unpleasant situation, where the recommended items are simply the ones similar to what they have purchased previously. One of the main reasons is that existing recommender systems in e-commerce mainly utilize primary implicit feedback (i.e., purchase history) for recommendation. Little attention has been paid to secondary implicit feedback (e.g., viewing items, adding items to shopping cart, adding items to favorite list, etc), which captures users' potential interests that may not be reflected in their purchase history. We therefore propose a personalized recommendation approach to combine the primary and secondary implicit feedback to generate the recommendation list, which is optimized towards a Bayesian objective criterion for personalized ranking. Experiments with a large-scale real-world e-commerce dataset show that the proposed approach presents a superior performance in comparison with the state-of-the-art baselines.
{"title":"Personalized Recommendation Considering Secondary Implicit Feedback","authors":"Siyuan Liu, Qiong Wu, C. Miao","doi":"10.1109/AGENTS.2018.8460053","DOIUrl":"https://doi.org/10.1109/AGENTS.2018.8460053","url":null,"abstract":"In e-commerce, recommendation is an essential feature to provide users with potentially interesting items to purchase. However, people are often faced with an unpleasant situation, where the recommended items are simply the ones similar to what they have purchased previously. One of the main reasons is that existing recommender systems in e-commerce mainly utilize primary implicit feedback (i.e., purchase history) for recommendation. Little attention has been paid to secondary implicit feedback (e.g., viewing items, adding items to shopping cart, adding items to favorite list, etc), which captures users' potential interests that may not be reflected in their purchase history. We therefore propose a personalized recommendation approach to combine the primary and secondary implicit feedback to generate the recommendation list, which is optimized towards a Bayesian objective criterion for personalized ranking. Experiments with a large-scale real-world e-commerce dataset show that the proposed approach presents a superior performance in comparison with the state-of-the-art baselines.","PeriodicalId":248901,"journal":{"name":"2018 IEEE International Conference on Agents (ICA)","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127982260","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-07-01DOI: 10.1109/AGENTS.2018.8460078
Chiao-Ting Chen, An-Pin Chen, Szu-Hao Huang
Investment decision making is considered as a series of complicated processes, which are difficult to be analyzed and imitated. Given large amounts of trading records with rich expert knowledge in financial domain, extracting its original decision logics and cloning the trading strategies are also quite challenging. In this paper, an agent-based reinforcement learning (RL) system is proposed to mimic professional trading strategies. The concept of continuous Markov decision process (MDP) in RL is similar to the trading decision making in financial time series data. With the specific-designed RL components, including states, actions, and rewards for financial applications, policy gradient method can successfully imitate the expert's strategies. In order to improve the convergence of RL agent in such highly dynamic environment, a pre-trained model based on supervised learning is transferred to the deep policy networks. The experimental results show that the proposed system can reproduce around eighty percent trading decisions both in training and testing stages. With the discussion of the tradeoff between explorations and model updating, this paper tried to fine-tuning the system parameters to get reasonable results. Finally, an advanced strategy is proposed to dynamically adjust the number of explorations in each episode to achieve better results.
{"title":"Cloning Strategies from Trading Records using Agent-based Reinforcement Learning Algorithm","authors":"Chiao-Ting Chen, An-Pin Chen, Szu-Hao Huang","doi":"10.1109/AGENTS.2018.8460078","DOIUrl":"https://doi.org/10.1109/AGENTS.2018.8460078","url":null,"abstract":"Investment decision making is considered as a series of complicated processes, which are difficult to be analyzed and imitated. Given large amounts of trading records with rich expert knowledge in financial domain, extracting its original decision logics and cloning the trading strategies are also quite challenging. In this paper, an agent-based reinforcement learning (RL) system is proposed to mimic professional trading strategies. The concept of continuous Markov decision process (MDP) in RL is similar to the trading decision making in financial time series data. With the specific-designed RL components, including states, actions, and rewards for financial applications, policy gradient method can successfully imitate the expert's strategies. In order to improve the convergence of RL agent in such highly dynamic environment, a pre-trained model based on supervised learning is transferred to the deep policy networks. The experimental results show that the proposed system can reproduce around eighty percent trading decisions both in training and testing stages. With the discussion of the tradeoff between explorations and model updating, this paper tried to fine-tuning the system parameters to get reasonable results. Finally, an advanced strategy is proposed to dynamically adjust the number of explorations in each episode to achieve better results.","PeriodicalId":248901,"journal":{"name":"2018 IEEE International Conference on Agents (ICA)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124166859","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-07-01DOI: 10.1109/AGENTS.2018.8460067
Peng Wang, W. Zhou, Di Wang, A. Tan
Exploration is essential in reinforcement learning, which expands the search space of potential solutions to a given problem for performance evaluations. Specifically, carefully designed exploration strategy may help the agent learn faster by taking the advantage of what it has learned previously. However, many reinforcement learning mechanisms still adopt simple exploration strategies, which select actions in a pure random manner among all the feasible actions. In this paper, we propose novel mechanisms to improve the existing knowledge-based exploration strategy based on a probabilistic guided approach to select actions. We conduct extensive experiments in a Minefield navigation simulator and the results show that our proposed probabilistic guided exploration approach significantly improves the convergence rate.
{"title":"Probabilistic Guided Exploration for Reinforcement Learning in Self-Organizing Neural Networks","authors":"Peng Wang, W. Zhou, Di Wang, A. Tan","doi":"10.1109/AGENTS.2018.8460067","DOIUrl":"https://doi.org/10.1109/AGENTS.2018.8460067","url":null,"abstract":"Exploration is essential in reinforcement learning, which expands the search space of potential solutions to a given problem for performance evaluations. Specifically, carefully designed exploration strategy may help the agent learn faster by taking the advantage of what it has learned previously. However, many reinforcement learning mechanisms still adopt simple exploration strategies, which select actions in a pure random manner among all the feasible actions. In this paper, we propose novel mechanisms to improve the existing knowledge-based exploration strategy based on a probabilistic guided approach to select actions. We conduct extensive experiments in a Minefield navigation simulator and the results show that our proposed probabilistic guided exploration approach significantly improves the convergence rate.","PeriodicalId":248901,"journal":{"name":"2018 IEEE International Conference on Agents (ICA)","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124373901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-07-01DOI: 10.1109/AGENTS.2018.8460127
Xun Tang, Takayuki Ito
Given the growing interest in automated negotiation, the search for effective strategies has produced a variety of different negotiation agents[8]. The Automated Negotiating Agents Competition(ANAC) has been annually held since 2010. The ANAC is an international competition that challenges researchers to design the agents that are able to operate effectively in different kinds of scenarios. In this competition, researchers analyze the negotiation agents from the aspects of utility, social welfare, distance to Nash solution, distance to Pareto and so on. Most of the analyses are based on negotiation results. Actually, the negotiation process can affect the negotiation results. To reach an agreement, agents analyze bids opponents have proposed and use a threshold function to decide how to compromise, this will determine the negotiation results. In this paper, we will describe a method called “negotiating efficiency” to evaluate the negotiation process. We will also explain how the process can affect the result.
{"title":"Metric for Evaluating Negotiation Process in Automated Negotiation","authors":"Xun Tang, Takayuki Ito","doi":"10.1109/AGENTS.2018.8460127","DOIUrl":"https://doi.org/10.1109/AGENTS.2018.8460127","url":null,"abstract":"Given the growing interest in automated negotiation, the search for effective strategies has produced a variety of different negotiation agents[8]. The Automated Negotiating Agents Competition(ANAC) has been annually held since 2010. The ANAC is an international competition that challenges researchers to design the agents that are able to operate effectively in different kinds of scenarios. In this competition, researchers analyze the negotiation agents from the aspects of utility, social welfare, distance to Nash solution, distance to Pareto and so on. Most of the analyses are based on negotiation results. Actually, the negotiation process can affect the negotiation results. To reach an agreement, agents analyze bids opponents have proposed and use a threshold function to decide how to compromise, this will determine the negotiation results. In this paper, we will describe a method called “negotiating efficiency” to evaluate the negotiation process. We will also explain how the process can affect the result.","PeriodicalId":248901,"journal":{"name":"2018 IEEE International Conference on Agents (ICA)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123433826","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-07-01DOI: 10.1109/AGENTS.2018.8460004
Zhepei Wei, D. Wang, M. Zhang, A. Tan, C. Miao, You Zhou
Since DeepMind pioneered a deep reinforcement learning (DRL) model to play the Atari games, DRL has become a commonly adopted method to enable the agents to learn complex control policies in various video games. However, similar approaches may still need to be improved when applied to more challenging scenarios, where reward signals are sparse and delayed. In this paper, we develop a refined DRL model to enable our autonomous agent to play the classical Snake Game, whose constraint gets stricter as the game progresses. Specifically, we employ a convolutional neural network (CNN) trained with a variant of Q-learning. Moreover, we propose a carefully designed reward mechanism to properly train the network, adopt a training gap strategy to temporarily bypass training after the location of the target changes, and introduce a dual experience replay method to categorize different experiences for better training efficacy. The experimental results show that our agent outperforms the baseline model and surpasses human-level performance in terms of playing the Snake Game.
{"title":"Autonomous Agents in Snake Game via Deep Reinforcement Learning","authors":"Zhepei Wei, D. Wang, M. Zhang, A. Tan, C. Miao, You Zhou","doi":"10.1109/AGENTS.2018.8460004","DOIUrl":"https://doi.org/10.1109/AGENTS.2018.8460004","url":null,"abstract":"Since DeepMind pioneered a deep reinforcement learning (DRL) model to play the Atari games, DRL has become a commonly adopted method to enable the agents to learn complex control policies in various video games. However, similar approaches may still need to be improved when applied to more challenging scenarios, where reward signals are sparse and delayed. In this paper, we develop a refined DRL model to enable our autonomous agent to play the classical Snake Game, whose constraint gets stricter as the game progresses. Specifically, we employ a convolutional neural network (CNN) trained with a variant of Q-learning. Moreover, we propose a carefully designed reward mechanism to properly train the network, adopt a training gap strategy to temporarily bypass training after the location of the target changes, and introduce a dual experience replay method to categorize different experiences for better training efficacy. The experimental results show that our agent outperforms the baseline model and surpasses human-level performance in terms of playing the Snake Game.","PeriodicalId":248901,"journal":{"name":"2018 IEEE International Conference on Agents (ICA)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122117192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-07-01DOI: 10.1109/agents.2018.8459969
{"title":"Proceedings: 2018 IEEE International Conference on Agents (ICA)","authors":"","doi":"10.1109/agents.2018.8459969","DOIUrl":"https://doi.org/10.1109/agents.2018.8459969","url":null,"abstract":"","PeriodicalId":248901,"journal":{"name":"2018 IEEE International Conference on Agents (ICA)","volume":"47 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"113959944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-07-01DOI: 10.1109/AGENTS.2018.8459932
Masahiro Nishi, Naoki Fukuta
In this paper, we show an analysis on a Misrepresentation Game with ambiguous preferences. A Misrepresentation Game is a game that sometimes an agent obtains higher utility than truth-telling on a preference-elicitation based fair division negotiation by misrepresenting their preferences while it is still difficult to be noticed by the counterpart. We investigate whether we can generate mechanisms for fair negotiations which avoids incentives to make misrepresentations by using a way of automated design of mechanisms.
{"title":"Toward a Misrepresentation Game with Ambiguous Preferences","authors":"Masahiro Nishi, Naoki Fukuta","doi":"10.1109/AGENTS.2018.8459932","DOIUrl":"https://doi.org/10.1109/AGENTS.2018.8459932","url":null,"abstract":"In this paper, we show an analysis on a Misrepresentation Game with ambiguous preferences. A Misrepresentation Game is a game that sometimes an agent obtains higher utility than truth-telling on a preference-elicitation based fair division negotiation by misrepresenting their preferences while it is still difficult to be noticed by the counterpart. We investigate whether we can generate mechanisms for fair negotiations which avoids incentives to make misrepresentations by using a way of automated design of mechanisms.","PeriodicalId":248901,"journal":{"name":"2018 IEEE International Conference on Agents (ICA)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125097663","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：