Pub Date : 2021-07-01DOI: 10.1109/ICCMA53594.2021.00019
Abel Yeboah-Ofori, Umar Mukhtar Ismail, Tymoteusz Swidurski, F. Opoku-Boateng
Cyberattacks on cyber supply chain (CSC) systems and the cascading impacts have brought many challenges and different threat levels with unpredictable consequences. The embedded networks nodes have various loopholes that could be exploited by the threat actors leading to various attacks, risks, and the threat of cascading attacks on the various systems. Key factors such as lack of common ontology vocabulary and semantic interoperability of cyberattack information, inadequate conceptualized ontology learning and hierarchical approach to representing the relationships in the CSC security domain has led to explicit knowledge representation. This paper explores cyberattack ontology learning to describe security concepts, properties and the relationships required to model security goal. Cyberattack ontology provides a semantic mapping between different organizational and vendor security goals has been inherently challenging. The contributions of this paper are threefold. First, we consider CSC security modelling such as goal, actor, attack, TTP, and requirements using semantic rules for logical representation. Secondly, we model a cyberattack ontology for semantic mapping and knowledge representation. Finally, we discuss concepts for threat intelligence and knowledge reuse. The results show that the cyberattack ontology concepts could be used to improve CSC security.
{"title":"Cyberattack Ontology: A Knowledge Representation for Cyber Supply Chain Security","authors":"Abel Yeboah-Ofori, Umar Mukhtar Ismail, Tymoteusz Swidurski, F. Opoku-Boateng","doi":"10.1109/ICCMA53594.2021.00019","DOIUrl":"https://doi.org/10.1109/ICCMA53594.2021.00019","url":null,"abstract":"Cyberattacks on cyber supply chain (CSC) systems and the cascading impacts have brought many challenges and different threat levels with unpredictable consequences. The embedded networks nodes have various loopholes that could be exploited by the threat actors leading to various attacks, risks, and the threat of cascading attacks on the various systems. Key factors such as lack of common ontology vocabulary and semantic interoperability of cyberattack information, inadequate conceptualized ontology learning and hierarchical approach to representing the relationships in the CSC security domain has led to explicit knowledge representation. This paper explores cyberattack ontology learning to describe security concepts, properties and the relationships required to model security goal. Cyberattack ontology provides a semantic mapping between different organizational and vendor security goals has been inherently challenging. The contributions of this paper are threefold. First, we consider CSC security modelling such as goal, actor, attack, TTP, and requirements using semantic rules for logical representation. Secondly, we model a cyberattack ontology for semantic mapping and knowledge representation. Finally, we discuss concepts for threat intelligence and knowledge reuse. The results show that the cyberattack ontology concepts could be used to improve CSC security.","PeriodicalId":131082,"journal":{"name":"2021 International Conference on Computing, Computational Modelling and Applications (ICCMA)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132995870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-01DOI: 10.1109/ICCMA53594.2021.00016
Hajarisena Razafimahatratra, Mihaela Ilie, Andrianjaka Miary Rapatsalahy, Thomas Mahatody, S. Ilie, N. Razafindrakoto
The organizations involved in software production have an interest in ensuring that the software developed conforms to a practice accepted by industry, namely compliance with standards, excellent quality control, efficiency and timeliness. Therefore, emergent design is an appropriate solution for acquiring higher quality software at low cost but meeting the production deadline. Indeed, it aims to reduce technical debt, accidental complexity as well as hyper-genericity contained in a software development process. In this article, we propose an approach supporting the modeling of emergent design, which is a way to make it easy for a designer to test the design quality of an object-oriented system. Our strategy is to define the necessary steps for the realization of emergent design within the software development process. Therefore, emergent design elements that make up our model are redundancy, readability, testability, cohesion, coupling and refactoring. As a case study, we instantiate in the proposed model a UML class diagram to test its design quality in order to improve the diagram if necessary. In order to validate our approach, we evaluated the performance of the UML class diagram obtained after the realization of emergent design proposed in this paper using the CM-Builder’s technique. The result of the performance evaluation with a high score of 93.61% concerning the accuracy of the proposed approach allows us to deduce that the method presented in this paper is essential for software designers in order to achieve a modular and easy to maintain system.
{"title":"An Emergent Design Approach to test the Quality of the Software System","authors":"Hajarisena Razafimahatratra, Mihaela Ilie, Andrianjaka Miary Rapatsalahy, Thomas Mahatody, S. Ilie, N. Razafindrakoto","doi":"10.1109/ICCMA53594.2021.00016","DOIUrl":"https://doi.org/10.1109/ICCMA53594.2021.00016","url":null,"abstract":"The organizations involved in software production have an interest in ensuring that the software developed conforms to a practice accepted by industry, namely compliance with standards, excellent quality control, efficiency and timeliness. Therefore, emergent design is an appropriate solution for acquiring higher quality software at low cost but meeting the production deadline. Indeed, it aims to reduce technical debt, accidental complexity as well as hyper-genericity contained in a software development process. In this article, we propose an approach supporting the modeling of emergent design, which is a way to make it easy for a designer to test the design quality of an object-oriented system. Our strategy is to define the necessary steps for the realization of emergent design within the software development process. Therefore, emergent design elements that make up our model are redundancy, readability, testability, cohesion, coupling and refactoring. As a case study, we instantiate in the proposed model a UML class diagram to test its design quality in order to improve the diagram if necessary. In order to validate our approach, we evaluated the performance of the UML class diagram obtained after the realization of emergent design proposed in this paper using the CM-Builder’s technique. The result of the performance evaluation with a high score of 93.61% concerning the accuracy of the proposed approach allows us to deduce that the method presented in this paper is essential for software designers in order to achieve a modular and easy to maintain system.","PeriodicalId":131082,"journal":{"name":"2021 International Conference on Computing, Computational Modelling and Applications (ICCMA)","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126459486","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-01DOI: 10.1109/ICCMA53594.2021.00023
Owusu Agyeman Antwi, Albert Osei Owusu, Joseph Wumboranaan Nanjo, Gabriel Blanton Gidisu, Dennis Sackey, Haruna Mohammed
This paper deals with the study of collocation base station and the risk associated in mounting and erecting base stations (towers). The deployment and positioning of the base stations in the same geographic area (collocation zone) has been a prime concern resulting in interference and high radiation exposure. To reduce cost and the risk of erecting this tower. The research aims at analyzing the collocated base transceiver and carefully exploring the dangers in erecting the base stations. Data were collected on Collocated base transceiver station by different operators in Korley klotte on a particular site as collocated base station, the location of each base transceiver station was captured in the study area using the Ghana Post GPS mobile application. However, this data collected were analyzed using Microsoft excel and Statistical Package for Social Scientist’s (SPSS). The findings of the study showed that the radiofrequency power density exposure hazard index in Korley Klotte was observed to be within the allowable 4.5w/m2 and 9w/m2 with corresponding frequencies of 10KHz and 30GHz exposure limit to the public as recommended by International Commission on Non-Ionizing Radiation Protection (ICNIRP).
{"title":"Analysis of Collocated Base Transceiver Stations And Associated Risks in Erecting Base Stations","authors":"Owusu Agyeman Antwi, Albert Osei Owusu, Joseph Wumboranaan Nanjo, Gabriel Blanton Gidisu, Dennis Sackey, Haruna Mohammed","doi":"10.1109/ICCMA53594.2021.00023","DOIUrl":"https://doi.org/10.1109/ICCMA53594.2021.00023","url":null,"abstract":"This paper deals with the study of collocation base station and the risk associated in mounting and erecting base stations (towers). The deployment and positioning of the base stations in the same geographic area (collocation zone) has been a prime concern resulting in interference and high radiation exposure. To reduce cost and the risk of erecting this tower. The research aims at analyzing the collocated base transceiver and carefully exploring the dangers in erecting the base stations. Data were collected on Collocated base transceiver station by different operators in Korley klotte on a particular site as collocated base station, the location of each base transceiver station was captured in the study area using the Ghana Post GPS mobile application. However, this data collected were analyzed using Microsoft excel and Statistical Package for Social Scientist’s (SPSS). The findings of the study showed that the radiofrequency power density exposure hazard index in Korley Klotte was observed to be within the allowable 4.5w/m2 and 9w/m2 with corresponding frequencies of 10KHz and 30GHz exposure limit to the public as recommended by International Commission on Non-Ionizing Radiation Protection (ICNIRP).","PeriodicalId":131082,"journal":{"name":"2021 International Conference on Computing, Computational Modelling and Applications (ICCMA)","volume":"824 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123291139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-01DOI: 10.1109/ICCMA53594.2021.00011
D. K. Dake, J. Gadze, G. S. Klogo
The emergence of 5G, IoT, Big Data, and related technologies have necessitated a shift to SDN architectural design and DRL algorithms for network task automation. Without prompt intelligent detection, the volumetric UDP flooding attack from zombies in an SDN-IoT network tends to consume network resources and mix with flash crowd events from legitimate hosts. This paper proposes a multiagent reinforcement learning framework in SDN-IoT to detect and mitigate DDoS attacks and route flash crowd events in the network effectively without compromising benign traffic. We simulated a 200 nodes topology with higher bandwidth and transmission rate in Mininet and implemented a multiagent deep deterministic policy gradient (MADDPG) algorithm for the framework. From the simulation results, the proposed approach outperforms Deep Deterministic Policy Gradient (DDPG) algorithm for the following network metrics: delay; jitter; packet loss; intrusion detection; and bandwidth utilization of network flows
{"title":"DDoS and Flash Event Detection in Higher Bandwidth SDN-IoT using Multiagent Reinforcement Learning","authors":"D. K. Dake, J. Gadze, G. S. Klogo","doi":"10.1109/ICCMA53594.2021.00011","DOIUrl":"https://doi.org/10.1109/ICCMA53594.2021.00011","url":null,"abstract":"The emergence of 5G, IoT, Big Data, and related technologies have necessitated a shift to SDN architectural design and DRL algorithms for network task automation. Without prompt intelligent detection, the volumetric UDP flooding attack from zombies in an SDN-IoT network tends to consume network resources and mix with flash crowd events from legitimate hosts. This paper proposes a multiagent reinforcement learning framework in SDN-IoT to detect and mitigate DDoS attacks and route flash crowd events in the network effectively without compromising benign traffic. We simulated a 200 nodes topology with higher bandwidth and transmission rate in Mininet and implemented a multiagent deep deterministic policy gradient (MADDPG) algorithm for the framework. From the simulation results, the proposed approach outperforms Deep Deterministic Policy Gradient (DDPG) algorithm for the following network metrics: delay; jitter; packet loss; intrusion detection; and bandwidth utilization of network flows","PeriodicalId":131082,"journal":{"name":"2021 International Conference on Computing, Computational Modelling and Applications (ICCMA)","volume":"125 17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127269732","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-01DOI: 10.1109/ICCMA53594.2021.00022
Emmanuel Kwabla Ocloo, R. Abubakar, G. Kumar
Challenges associated with borrowing have become a worldly topic worth researching. Most especially in sub-Saharan Africa due to the poverty rate which is associated with lack of capital to state businesses. Thus, this study seeks to investigate the issues with borrowing and its underlying effects on Micro Small and Medium Enterprise (MSMEs) in Kaneshie in the Greater Accra Region of Ghana. The entire population targeted for this research was 73 and same were sampled. But, 65 respondents were randomly drawn out of the 73 population for a mixed research approach.Some of the challenges associated with borrowing by MSMEs are collateral requirement, tedious loan procedures and long disbursal periods. Also, the effects established on borrowed funds are high interest rates, tax, inflation as well as the risk association.The research employed descriptive research design since it portrays accurate responses from owners that are targeted and MSMEs’ managers in the targeted population. Structured questionnaires were used to collect primary data through direct interview of entrepreneurs and governing body officials. Both qualitative and quantitative methods of data collection were applied in this study. Also, data collected was analyzed using Statistical Package for Social Sciences (SPSS). The findings of this research would assist MSMEs in selecting the appropriate funding sources. The study would also serve as a learning tool for students as well as a tool for entrepreneurial business development.
{"title":"Assessment of Challenges with Borrowing and Underlying Effects of MSMEs in Kaneshie, Greater Accra Region of Ghana","authors":"Emmanuel Kwabla Ocloo, R. Abubakar, G. Kumar","doi":"10.1109/ICCMA53594.2021.00022","DOIUrl":"https://doi.org/10.1109/ICCMA53594.2021.00022","url":null,"abstract":"Challenges associated with borrowing have become a worldly topic worth researching. Most especially in sub-Saharan Africa due to the poverty rate which is associated with lack of capital to state businesses. Thus, this study seeks to investigate the issues with borrowing and its underlying effects on Micro Small and Medium Enterprise (MSMEs) in Kaneshie in the Greater Accra Region of Ghana. The entire population targeted for this research was 73 and same were sampled. But, 65 respondents were randomly drawn out of the 73 population for a mixed research approach.Some of the challenges associated with borrowing by MSMEs are collateral requirement, tedious loan procedures and long disbursal periods. Also, the effects established on borrowed funds are high interest rates, tax, inflation as well as the risk association.The research employed descriptive research design since it portrays accurate responses from owners that are targeted and MSMEs’ managers in the targeted population. Structured questionnaires were used to collect primary data through direct interview of entrepreneurs and governing body officials. Both qualitative and quantitative methods of data collection were applied in this study. Also, data collected was analyzed using Statistical Package for Social Sciences (SPSS). The findings of this research would assist MSMEs in selecting the appropriate funding sources. The study would also serve as a learning tool for students as well as a tool for entrepreneurial business development.","PeriodicalId":131082,"journal":{"name":"2021 International Conference on Computing, Computational Modelling and Applications (ICCMA)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123635805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-01DOI: 10.1109/ICCMA53594.2021.00012
A. Dixit, Shreya Oak, Shaney Mantri, Anant V. Nimkar, Meghana Naik
With the ever burgeoning traffic and increasing unwarranted lane changes, safety of commuters and driving practices have been compromised. Most of the accidents that take place are due to unforeseen traffic and road conditions or misjudgement by the drivers. To aid the decision making of drivers to maneuver the vehicle safely, this paper proposes ELaNES, a novel system focusing on the lane changing behaviour of a vehicle. It posits a method to predict the lane change and state the characteristics associated with it. Along with this, a degree of safe lane change is calculated by taking into consideration the neighbouring vehicles’ dynamics and the driver’s intentions. The results obtained are promising, accurate and can help make decisions in the real world scenario, along with giving deep insights into human driving practices.
{"title":"Lane Change Prediction based on Trajectory Analysis and Driver Actions Tracking","authors":"A. Dixit, Shreya Oak, Shaney Mantri, Anant V. Nimkar, Meghana Naik","doi":"10.1109/ICCMA53594.2021.00012","DOIUrl":"https://doi.org/10.1109/ICCMA53594.2021.00012","url":null,"abstract":"With the ever burgeoning traffic and increasing unwarranted lane changes, safety of commuters and driving practices have been compromised. Most of the accidents that take place are due to unforeseen traffic and road conditions or misjudgement by the drivers. To aid the decision making of drivers to maneuver the vehicle safely, this paper proposes ELaNES, a novel system focusing on the lane changing behaviour of a vehicle. It posits a method to predict the lane change and state the characteristics associated with it. Along with this, a degree of safe lane change is calculated by taking into consideration the neighbouring vehicles’ dynamics and the driver’s intentions. The results obtained are promising, accurate and can help make decisions in the real world scenario, along with giving deep insights into human driving practices.","PeriodicalId":131082,"journal":{"name":"2021 International Conference on Computing, Computational Modelling and Applications (ICCMA)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126620963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-01DOI: 10.1109/ICCMA53594.2021.00034
Q. Kester, Emeh Jennifer Afoma
If financial institutions cannot detect incidents effectively, it cannot succeed in responding to incidents. This implies that the detection of incidents, is the most important aspect of incident response. A stochastic process with a first order dependence in discrete state and time is described as Markov chain, in the same way, Bayesian theory is a mathematical framework for reasoning and performing inference using probability. These two theories when based on socioeconomic factors can be used to predict cybercrime occurrence in Management Information Systems (MIS). The advancement of technology in banking has made banking business processes very convenient, but as the technology advances, cybercrimes of different nature emerges and equally at its peak. In as much as there are different measures already in place to combat these crimes, there still lies so many vulnerabilities which cannot be evitable in any information systems. Financial institutions need to develop predictive models that can be used to combat this cybercrime activities.In this paper, the application of Markov chain and Bayesian inference was used to analyze the nature of cybercrime and the probability of its occurrence, and the results were used to analyze the possibility of occurrence of cybercrimes based on the factors considered.
{"title":"Crime Predictive Model in Cybercrime based on Social and Economic Factors Using the Bayesian and Markov Theories","authors":"Q. Kester, Emeh Jennifer Afoma","doi":"10.1109/ICCMA53594.2021.00034","DOIUrl":"https://doi.org/10.1109/ICCMA53594.2021.00034","url":null,"abstract":"If financial institutions cannot detect incidents effectively, it cannot succeed in responding to incidents. This implies that the detection of incidents, is the most important aspect of incident response. A stochastic process with a first order dependence in discrete state and time is described as Markov chain, in the same way, Bayesian theory is a mathematical framework for reasoning and performing inference using probability. These two theories when based on socioeconomic factors can be used to predict cybercrime occurrence in Management Information Systems (MIS). The advancement of technology in banking has made banking business processes very convenient, but as the technology advances, cybercrimes of different nature emerges and equally at its peak. In as much as there are different measures already in place to combat these crimes, there still lies so many vulnerabilities which cannot be evitable in any information systems. Financial institutions need to develop predictive models that can be used to combat this cybercrime activities.In this paper, the application of Markov chain and Bayesian inference was used to analyze the nature of cybercrime and the probability of its occurrence, and the results were used to analyze the possibility of occurrence of cybercrimes based on the factors considered.","PeriodicalId":131082,"journal":{"name":"2021 International Conference on Computing, Computational Modelling and Applications (ICCMA)","volume":"111 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121741760","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-06-10DOI: 10.1109/ICCMA53594.2021.00018
Vincent Zha, Ivey Chiu, Alexandre Guilbault, Jaime Tatis
Slowly changing variables in a continuous state space constitute an important category of reinforcement learning and see their applications in many domains, such as modeling a climate control system where temperature, humidity, etc. change slowly over time. However, this subject is less addressed in relevant studies. Classical methods with certain variants, such as Dynamic Programming with Tile Coding which discretizes the state space, fail to handle slowly changing variables because those methods cannot capture the tiny changes in each transition step, as it is computationally expensive or impossible to establish an extremely granular grid system. In this paper, we introduce a Hyperspace Neighbor Penetration (HNP) approach that solves the problem. HNP captures in each transition step the state’s partial “penetration” into its neighboring hyper-tiles in the gridded hyperspace, thus does not require the transition to be inter-tile for the change to be captured. Therefore, HNP allows for a very coarse grid system, which makes the computation feasible. HNP assumes near linearity of the transition function in a local space, which is commonly satisfied. In summary, HNP can be orders of magnitude more efficient than classical method in handling slowly changing variables in reinforcement learning. We have successfully made an industrial implementation of NHP.
连续状态空间中缓慢变化的变量构成了强化学习的一个重要类别，并在许多领域中得到应用，例如对温度、湿度等随时间缓慢变化的气候控制系统进行建模。然而，这一主题在相关研究中较少涉及。具有某些变量的经典方法，如离散状态空间的动态编程(Dynamic Programming with Tile Coding)，无法处理缓慢变化的变量，因为这些方法无法捕捉到每个过渡步骤中的微小变化，因为它的计算成本很高，或者不可能建立一个极细粒度的网格系统。在本文中，我们引入了一种超空间邻居渗透(HNP)方法来解决这个问题。HNP在每个转换步骤中捕获状态对网格超空间中相邻超块的部分“渗透”，因此不需要转换是块间的即可捕获更改。因此，HNP允许一个非常粗糙的网格系统，这使得计算可行。HNP假设过渡函数在局部空间近似线性，一般满足这一条件。总之，在处理强化学习中缓慢变化的变量时，HNP可以比经典方法效率高几个数量级。我们已经成功地实现了NHP的工业实施。
{"title":"Hyperspace Neighbor Penetration Approach to Dynamic Programming for Model-Based Reinforcement Learning Problems with Slowly Changing Variables in a Continuous State Space","authors":"Vincent Zha, Ivey Chiu, Alexandre Guilbault, Jaime Tatis","doi":"10.1109/ICCMA53594.2021.00018","DOIUrl":"https://doi.org/10.1109/ICCMA53594.2021.00018","url":null,"abstract":"Slowly changing variables in a continuous state space constitute an important category of reinforcement learning and see their applications in many domains, such as modeling a climate control system where temperature, humidity, etc. change slowly over time. However, this subject is less addressed in relevant studies. Classical methods with certain variants, such as Dynamic Programming with Tile Coding which discretizes the state space, fail to handle slowly changing variables because those methods cannot capture the tiny changes in each transition step, as it is computationally expensive or impossible to establish an extremely granular grid system. In this paper, we introduce a Hyperspace Neighbor Penetration (HNP) approach that solves the problem. HNP captures in each transition step the state’s partial “penetration” into its neighboring hyper-tiles in the gridded hyperspace, thus does not require the transition to be inter-tile for the change to be captured. Therefore, HNP allows for a very coarse grid system, which makes the computation feasible. HNP assumes near linearity of the transition function in a local space, which is commonly satisfied. In summary, HNP can be orders of magnitude more efficient than classical method in handling slowly changing variables in reinforcement learning. We have successfully made an industrial implementation of NHP.","PeriodicalId":131082,"journal":{"name":"2021 International Conference on Computing, Computational Modelling and Applications (ICCMA)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126655197","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：