Pub Date : 2022-10-12DOI: 10.1109/AICT55583.2022.10013578
Jacopo Fior, Luca Cagliero
Reinforcement Learning techniques have shown a great potential in the active allocation of stock portfolios. However, state-of-the-art solutions show limited stability and fairly high sensitivity to volatile market conditions. To tackle these issues, this paper presents a new risk-aware approach based on Deep Q-learning Networks. It leverages Quantile Regression DQNs to mitigate the underlying market risks and an action branching architecture to effectively handle high-dimensional stock spaces. Furthermore, it also introduces noise perturbations to the network’s weights aimed at self-tuning the degree of exploration for each input dimension. Based on the empirical simulations, which were carried out on the Dow Jones-30 stocks over a three-year period, the proposed system performs better than state-of-the-art RL solutions in terms of cumulative return, stability, and sharpe ratio.
{"title":"A risk-aware approach to stock portfolio allocation based on Deep Q-Networks","authors":"Jacopo Fior, Luca Cagliero","doi":"10.1109/AICT55583.2022.10013578","DOIUrl":"https://doi.org/10.1109/AICT55583.2022.10013578","url":null,"abstract":"Reinforcement Learning techniques have shown a great potential in the active allocation of stock portfolios. However, state-of-the-art solutions show limited stability and fairly high sensitivity to volatile market conditions. To tackle these issues, this paper presents a new risk-aware approach based on Deep Q-learning Networks. It leverages Quantile Regression DQNs to mitigate the underlying market risks and an action branching architecture to effectively handle high-dimensional stock spaces. Furthermore, it also introduces noise perturbations to the network’s weights aimed at self-tuning the degree of exploration for each input dimension. Based on the empirical simulations, which were carried out on the Dow Jones-30 stocks over a three-year period, the proposed system performs better than state-of-the-art RL solutions in terms of cumulative return, stability, and sharpe ratio.","PeriodicalId":441475,"journal":{"name":"2022 IEEE 16th International Conference on Application of Information and Communication Technologies (AICT)","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132981449","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-12DOI: 10.1109/AICT55583.2022.10013627
F. Abdullayeva, Kamala Gurbanova
The dynamic development of computer technology and means of communication and the improvement of network technology have led to an increase in the role of information as a major resource in society. People with hearing impairments, like everyone else, need to benefit from all areas where ICT is applied. Gestures are the only way for people with hearing and speech disabilities to communicate. Automatic recognition of gestures to facilitate communication with gestures is a topical issue, both scientifically and practically. The study provides information on static and dynamic gestures, various sensor technologies used in the collection of gesture data have been researched. The advantages and disadvantages of image-based and non-image-based technologies are analysed. A machine learning method based on neural networks has been developed for high-precision identification of gestures. High results were obtained when testing the developed method on a database open to scientific research. Thus, the method was able to recognize the letters of the dactyl alphabet with an accuracy of 0.95, 0.92, 0.95, 0.94 on the indicators of accuracy, precision, recall, F1-score, respectively.
{"title":"Sign Language Hand Gesture Recognition Method based on Machine Learning","authors":"F. Abdullayeva, Kamala Gurbanova","doi":"10.1109/AICT55583.2022.10013627","DOIUrl":"https://doi.org/10.1109/AICT55583.2022.10013627","url":null,"abstract":"The dynamic development of computer technology and means of communication and the improvement of network technology have led to an increase in the role of information as a major resource in society. People with hearing impairments, like everyone else, need to benefit from all areas where ICT is applied. Gestures are the only way for people with hearing and speech disabilities to communicate. Automatic recognition of gestures to facilitate communication with gestures is a topical issue, both scientifically and practically. The study provides information on static and dynamic gestures, various sensor technologies used in the collection of gesture data have been researched. The advantages and disadvantages of image-based and non-image-based technologies are analysed. A machine learning method based on neural networks has been developed for high-precision identification of gestures. High results were obtained when testing the developed method on a database open to scientific research. Thus, the method was able to recognize the letters of the dactyl alphabet with an accuracy of 0.95, 0.92, 0.95, 0.94 on the indicators of accuracy, precision, recall, F1-score, respectively.","PeriodicalId":441475,"journal":{"name":"2022 IEEE 16th International Conference on Application of Information and Communication Technologies (AICT)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132455936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-12DOI: 10.1109/AICT55583.2022.10013619
Grigory Kulagin, Ivan Ermakov, L. Lyadova
The quality of the systems depends on compliance to the domain requirements. High quality is achieved only with involving experts in the relevant fields to the system design as experts. Modern design methods are based on using professional tools and modeling languages. Using these tools are difficult for domain experts. Domain-Specific Languages (DSLs) can be considered as "user interfaces" for experts because they bridge the gap between the domain experts and the software development tools via customizing modeling languages. Usability of DSLs by domain experts is a key factor for their successful adoption. But DSL creation is challenging task. An approach to DSL customization based on using multifaceted ontology is proposed. General scheme of DSL metamodel generation based on multifaceted ontology is described. Examples of created DSLs and models illustrating the applicability of the proposed method are shown. The DSL metamodels were developed and tested in several domains. The results of experiments confirmed practical significance of the ontology-based approach to DSL creation.
{"title":"Ontology-Based Development of Domain-Specific Languages via Customizing Base Language","authors":"Grigory Kulagin, Ivan Ermakov, L. Lyadova","doi":"10.1109/AICT55583.2022.10013619","DOIUrl":"https://doi.org/10.1109/AICT55583.2022.10013619","url":null,"abstract":"The quality of the systems depends on compliance to the domain requirements. High quality is achieved only with involving experts in the relevant fields to the system design as experts. Modern design methods are based on using professional tools and modeling languages. Using these tools are difficult for domain experts. Domain-Specific Languages (DSLs) can be considered as \"user interfaces\" for experts because they bridge the gap between the domain experts and the software development tools via customizing modeling languages. Usability of DSLs by domain experts is a key factor for their successful adoption. But DSL creation is challenging task. An approach to DSL customization based on using multifaceted ontology is proposed. General scheme of DSL metamodel generation based on multifaceted ontology is described. Examples of created DSLs and models illustrating the applicability of the proposed method are shown. The DSL metamodels were developed and tested in several domains. The results of experiments confirmed practical significance of the ontology-based approach to DSL creation.","PeriodicalId":441475,"journal":{"name":"2022 IEEE 16th International Conference on Application of Information and Communication Technologies (AICT)","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122953486","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-12DOI: 10.1109/AICT55583.2022.10013649
Kyung-yong Lee, Jae-Seon Jang
It is important to acquire accurate synchronization in the time division duplex (TDD) system that uses downlink and uplink separately in time because interference exists when downlink and uplink time slots overlap. It is also important to know the precise timing of downlink and uplink for the 5G RF repeater that receives the signal of the base station with the link antenna, amplifies it, and serves through a service antenna because the 5G RF repeater also switches the roles of link antenna and service antenna when switching downlink and uplink. The synchronization signal block (SSB) decoding method is used to know the downlink and uplink timings of the base station. However, this method has the disadvantage that it cannot be used when the SS signal to interference and noise ratio (SS-SINR) is below the decodable SINR such as the boundary of multiple cells. This paper proposes a new method of synchronization signal abstraction that can acquire synchronization even in the section where SS-SINR is below the decodable SINR due to SS-SINR interference by introducing a way to acquire synchronization without decoding by receiving SSB and identifying its pattern. The new method of synchronization signal abstraction can increase the installation coverage of the 5G RF repeater by 293% compared to the SSB decoding method.
{"title":"Novel method for synthesis of synchronization signal abstraction for 5G RF repeater","authors":"Kyung-yong Lee, Jae-Seon Jang","doi":"10.1109/AICT55583.2022.10013649","DOIUrl":"https://doi.org/10.1109/AICT55583.2022.10013649","url":null,"abstract":"It is important to acquire accurate synchronization in the time division duplex (TDD) system that uses downlink and uplink separately in time because interference exists when downlink and uplink time slots overlap. It is also important to know the precise timing of downlink and uplink for the 5G RF repeater that receives the signal of the base station with the link antenna, amplifies it, and serves through a service antenna because the 5G RF repeater also switches the roles of link antenna and service antenna when switching downlink and uplink. The synchronization signal block (SSB) decoding method is used to know the downlink and uplink timings of the base station. However, this method has the disadvantage that it cannot be used when the SS signal to interference and noise ratio (SS-SINR) is below the decodable SINR such as the boundary of multiple cells. This paper proposes a new method of synchronization signal abstraction that can acquire synchronization even in the section where SS-SINR is below the decodable SINR due to SS-SINR interference by introducing a way to acquire synchronization without decoding by receiving SSB and identifying its pattern. The new method of synchronization signal abstraction can increase the installation coverage of the 5G RF repeater by 293% compared to the SSB decoding method.","PeriodicalId":441475,"journal":{"name":"2022 IEEE 16th International Conference on Application of Information and Communication Technologies (AICT)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131293949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-12DOI: 10.1109/AICT55583.2022.10013575
M. Mammadova, Zarifa Jabrayilova, Lala Karayeva, A. Ahmadova
The prevention of hepatocellular carcinoma (HCC), which is rated third for causing death due to cancer in the world, and the selection of more effective treatment have necessitated the development of HCC diagnosis and prediction systems using artificial intelligence. The presented paper examines the possibility of applying machine learning algorithms to predict liver cancer. Machine learning methods such as Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF) are used to predict HCC. The HCC Dataset taken from the website Kaggle (Kaggle.com) is referenced for the realization of prediction. This research uses the libraries scikit- learn, Pandas, NumPy, etc. in the Jupiter programming environment to conduct experiments. The results of the experiments are compared, and the RF classifier is estimated to perform the highest result. Referring to this fact, the importance of using the RF method in building an initial HCC diagnosis and prognosis system is justified.
{"title":"Prediction of hepatocellular carcinoma using a machine learning algorithm","authors":"M. Mammadova, Zarifa Jabrayilova, Lala Karayeva, A. Ahmadova","doi":"10.1109/AICT55583.2022.10013575","DOIUrl":"https://doi.org/10.1109/AICT55583.2022.10013575","url":null,"abstract":"The prevention of hepatocellular carcinoma (HCC), which is rated third for causing death due to cancer in the world, and the selection of more effective treatment have necessitated the development of HCC diagnosis and prediction systems using artificial intelligence. The presented paper examines the possibility of applying machine learning algorithms to predict liver cancer. Machine learning methods such as Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF) are used to predict HCC. The HCC Dataset taken from the website Kaggle (Kaggle.com) is referenced for the realization of prediction. This research uses the libraries scikit- learn, Pandas, NumPy, etc. in the Jupiter programming environment to conduct experiments. The results of the experiments are compared, and the RF classifier is estimated to perform the highest result. Referring to this fact, the importance of using the RF method in building an initial HCC diagnosis and prognosis system is justified.","PeriodicalId":441475,"journal":{"name":"2022 IEEE 16th International Conference on Application of Information and Communication Technologies (AICT)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125262556","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-12DOI: 10.1109/AICT55583.2022.10013622
J. Hasanov
This paper reviews one of the contemporary problems in brain cancer diagnosis in its early stages, based on the hypothetical correlation of visual brain scans with the MGMT promoter methylation status. The analysis of the topics covers the methods of extracting features and evaluating their informative value, the challenges of working with the corresponding image samples, and models that might be useful in the given domain. The paper analyzes the possible technical solutions and discusses the efficiency of the domain-specific implementation. The experimental results are followed by the outcomes and suggestions for improvement.
{"title":"On approaches of investigating multi-layered medical images for implicit features","authors":"J. Hasanov","doi":"10.1109/AICT55583.2022.10013622","DOIUrl":"https://doi.org/10.1109/AICT55583.2022.10013622","url":null,"abstract":"This paper reviews one of the contemporary problems in brain cancer diagnosis in its early stages, based on the hypothetical correlation of visual brain scans with the MGMT promoter methylation status. The analysis of the topics covers the methods of extracting features and evaluating their informative value, the challenges of working with the corresponding image samples, and models that might be useful in the given domain. The paper analyzes the possible technical solutions and discusses the efficiency of the domain-specific implementation. The experimental results are followed by the outcomes and suggestions for improvement.","PeriodicalId":441475,"journal":{"name":"2022 IEEE 16th International Conference on Application of Information and Communication Technologies (AICT)","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125088352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-12DOI: 10.1109/AICT55583.2022.10013558
F. M. Nazarov, Yarmatov Sherzodjon Shokir o’g’li, Eshtemirov Bunyod Sherali o’g’li
Currently, the transmission of data in the form of video and audio on computer networks is growing rapidly. Increasing the flow of media data makes it the task of qualitative data processing and analysis. The growth of video data is one of the most pressing issues in the intellectual analysis of the content of this video data and transcription to extract the texts contained in the video files. In the process of video transcription, the process of transferring the data in the video file to the text without errors is followed. The process of video transcription is important for the intellectual analysis of data in hearing-impaired and large- scale video networks. To date, a lot of research has been done on video transcription. Nevertheless, shortcomings in transcription remain sufficient for arbitrary language. This research includes theoretical research on the study of algorithms and models of video transcription, as well as a theoretical analysis of experiments based on them.
{"title":"Algorithms To Increase Data Reliability In Video Transcription","authors":"F. M. Nazarov, Yarmatov Sherzodjon Shokir o’g’li, Eshtemirov Bunyod Sherali o’g’li","doi":"10.1109/AICT55583.2022.10013558","DOIUrl":"https://doi.org/10.1109/AICT55583.2022.10013558","url":null,"abstract":"Currently, the transmission of data in the form of video and audio on computer networks is growing rapidly. Increasing the flow of media data makes it the task of qualitative data processing and analysis. The growth of video data is one of the most pressing issues in the intellectual analysis of the content of this video data and transcription to extract the texts contained in the video files. In the process of video transcription, the process of transferring the data in the video file to the text without errors is followed. The process of video transcription is important for the intellectual analysis of data in hearing-impaired and large- scale video networks. To date, a lot of research has been done on video transcription. Nevertheless, shortcomings in transcription remain sufficient for arbitrary language. This research includes theoretical research on the study of algorithms and models of video transcription, as well as a theoretical analysis of experiments based on them.","PeriodicalId":441475,"journal":{"name":"2022 IEEE 16th International Conference on Application of Information and Communication Technologies (AICT)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127443051","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-12DOI: 10.1109/AICT55583.2022.10013495
Junbing Li, Xiaoping Zeng, Guojun Li, Congji Yin, Chenxi Bai
The High Frequency (HF) multi-sideband transmission technology combines multiple 3KHz channels, which effectively improves the transmission rate of HF communication system, but also inevitably brings the defect of high peak-to-average-power ratio (PAPR). In this paper, a low-complexity blind selected mapping (SLM) algorithm for HF multi-sideband systems is proposed. Cyclic redundancy check (CRC) is performed on the received signal during phase recovery, and the result of the checksum can be used to determine whether the current signal is correctly recovered in phase. The expressions for the number of iterative computations are derived and the simulation results show that low-complexity blind SLM algorithm effectively reduces the computational complexity without increasing the bit error rate(BER).
{"title":"Low Complexity Blind SLM for PAPR reduction of HF XL systems","authors":"Junbing Li, Xiaoping Zeng, Guojun Li, Congji Yin, Chenxi Bai","doi":"10.1109/AICT55583.2022.10013495","DOIUrl":"https://doi.org/10.1109/AICT55583.2022.10013495","url":null,"abstract":"The High Frequency (HF) multi-sideband transmission technology combines multiple 3KHz channels, which effectively improves the transmission rate of HF communication system, but also inevitably brings the defect of high peak-to-average-power ratio (PAPR). In this paper, a low-complexity blind selected mapping (SLM) algorithm for HF multi-sideband systems is proposed. Cyclic redundancy check (CRC) is performed on the received signal during phase recovery, and the result of the checksum can be used to determine whether the current signal is correctly recovered in phase. The expressions for the number of iterative computations are derived and the simulation results show that low-complexity blind SLM algorithm effectively reduces the computational complexity without increasing the bit error rate(BER).","PeriodicalId":441475,"journal":{"name":"2022 IEEE 16th International Conference on Application of Information and Communication Technologies (AICT)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126927763","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-12DOI: 10.1109/AICT55583.2022.10013633
I. Jumanov, R. Safarov, O. Djumanov
The problem of visualization, recognition, classification of images of micro-objects, in particular, pollen grains, unicellular organisms, fingerprints based on the definition of their variety, belonging to a class, the use of information of geometric shapes, morphology, dynamic, specific characteristics, unique features of neural networks has been investigated, in control systems of industrial and technological complexes, environmental monitoring, ecology, and medical diagnoses. Methods, learning algorithms, component computational schemes of neural networks have been developed, which provide the best quality of image identification in conditions of a priori insufficiency, uncertainty of parameters, and low accuracy of data processing. Mathematical expressions are obtained for estimating identification errors associated with information distortions at the measurement, input, and transmission stages due to nonstationarity, the inadequacy of approximation, interpolation, and extrapolation of the image contour. A software package for the recognition and classification of pollen grains has been built and implemented, which includes algorithms for a three-layer, loosely coupled neural network, Hopfield’s network, bidirectional associative memory, Kohonen. Results are obtained for correct, incorrect recognition, and rejected pollen samples based on with-teacher and unsupervised learning algorithms, which are synthesized with cubic, biquadratic, and interpolation spline functions.
{"title":"Mechanisms For Using Image Properties And Neural Networks In Identification Of Micro-Objects","authors":"I. Jumanov, R. Safarov, O. Djumanov","doi":"10.1109/AICT55583.2022.10013633","DOIUrl":"https://doi.org/10.1109/AICT55583.2022.10013633","url":null,"abstract":"The problem of visualization, recognition, classification of images of micro-objects, in particular, pollen grains, unicellular organisms, fingerprints based on the definition of their variety, belonging to a class, the use of information of geometric shapes, morphology, dynamic, specific characteristics, unique features of neural networks has been investigated, in control systems of industrial and technological complexes, environmental monitoring, ecology, and medical diagnoses. Methods, learning algorithms, component computational schemes of neural networks have been developed, which provide the best quality of image identification in conditions of a priori insufficiency, uncertainty of parameters, and low accuracy of data processing. Mathematical expressions are obtained for estimating identification errors associated with information distortions at the measurement, input, and transmission stages due to nonstationarity, the inadequacy of approximation, interpolation, and extrapolation of the image contour. A software package for the recognition and classification of pollen grains has been built and implemented, which includes algorithms for a three-layer, loosely coupled neural network, Hopfield’s network, bidirectional associative memory, Kohonen. Results are obtained for correct, incorrect recognition, and rejected pollen samples based on with-teacher and unsupervised learning algorithms, which are synthesized with cubic, biquadratic, and interpolation spline functions.","PeriodicalId":441475,"journal":{"name":"2022 IEEE 16th International Conference on Application of Information and Communication Technologies (AICT)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123787069","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-12DOI: 10.1109/AICT55583.2022.10013603
Md. Rafat Rahman Tushar, Shahnewaz Siddique
To perform well, Deep Reinforcement Learning (DRL) methods require significant memory resources and computational time. Also, sometimes these systems need additional environment information to achieve a good reward. However, it is more important for many applications and devices to reduce memory usage and computational times than to achieve the maximum reward. This paper presents a modified DRL method that performs reasonably well with compressed imagery data without requiring additional environment information and also uses less memory and time. We have designed a lightweight Convolutional Neural Network (CNN) with a variant of the Q-network that efficiently takes preprocessed image data as input and uses less memory. Furthermore, we use a simple reward mechanism and small experience replay memory so as to provide only the minimum necessary information. Our modified DRL method enables our autonomous agent to play Snake, a classical control game. The results show our model can achieve similar performance as other DRL methods.
{"title":"A Memory Efficient Deep Reinforcement Learning Approach For Snake Game Autonomous Agents","authors":"Md. Rafat Rahman Tushar, Shahnewaz Siddique","doi":"10.1109/AICT55583.2022.10013603","DOIUrl":"https://doi.org/10.1109/AICT55583.2022.10013603","url":null,"abstract":"To perform well, Deep Reinforcement Learning (DRL) methods require significant memory resources and computational time. Also, sometimes these systems need additional environment information to achieve a good reward. However, it is more important for many applications and devices to reduce memory usage and computational times than to achieve the maximum reward. This paper presents a modified DRL method that performs reasonably well with compressed imagery data without requiring additional environment information and also uses less memory and time. We have designed a lightweight Convolutional Neural Network (CNN) with a variant of the Q-network that efficiently takes preprocessed image data as input and uses less memory. Furthermore, we use a simple reward mechanism and small experience replay memory so as to provide only the minimum necessary information. Our modified DRL method enables our autonomous agent to play Snake, a classical control game. The results show our model can achieve similar performance as other DRL methods.","PeriodicalId":441475,"journal":{"name":"2022 IEEE 16th International Conference on Application of Information and Communication Technologies (AICT)","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117227669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：