With the rapid development of the automotive industry, vehicles are equipped with a variety of system functions. The realization of many of these functions depends on the stable, safe, and reliable positioning and timing information of the bottom layer. Vehicle-mounted satellite positioning systems are an important way for vehicles to obtain absolute positions and have been widely employed in other industries. However, the automotive industry has its special requirements, such as high positioning accuracy and confidence, extremely harsh vehicle regulations, reliability, and high safety, all of which need to be tested and evaluated on vehicle-mounted satellite positioning systems. By studying and putting forward the evaluation scheme for the vehicle-mounted satellite positioning system, this paper further ensures the accuracy, reliability, and stability of the time-space information provided by the system and supports the development of the automotive industry.
{"title":"Research on test and evaluation scheme for vehicle-mounted satellite positioning receiver","authors":"Guotian Ji, Hui Rong, Tong Zhu, Fujiangshan He, Guokai Jiang","doi":"10.1117/12.2667205","DOIUrl":"https://doi.org/10.1117/12.2667205","url":null,"abstract":"With the rapid development of the automotive industry, vehicles are equipped with a variety of system functions. The realization of many of these functions depends on the stable, safe, and reliable positioning and timing information of the bottom layer. Vehicle-mounted satellite positioning systems are an important way for vehicles to obtain absolute positions and have been widely employed in other industries. However, the automotive industry has its special requirements, such as high positioning accuracy and confidence, extremely harsh vehicle regulations, reliability, and high safety, all of which need to be tested and evaluated on vehicle-mounted satellite positioning systems. By studying and putting forward the evaluation scheme for the vehicle-mounted satellite positioning system, this paper further ensures the accuracy, reliability, and stability of the time-space information provided by the system and supports the development of the automotive industry.","PeriodicalId":137914,"journal":{"name":"International Conference on Artificial Intelligence, Virtual Reality, and Visualization","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127119838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Wildfire, also known as forest fire, is fire that usually occur in forests and are difficult to control. If it could be detected and suppressed at an early stage (mainly smoke and flames), it has important meaning for reducing the loss. With the attention of relevant researchers, wildfire detection technology has become more and more advanced, from traditional manual monitoring to traditional target detection to sensor detection and infrared detection, etc. The various detection methods involved still have problems such as slow detection speed, low accuracy, easy interference and high cost. In this paper, SSD, an advanced target detection method, was chosen from deep learning algorithms. Three independent SSD networks are built with VGG16, MobileNet v2, and EfficientNet b3 as the backbone. The experimental results show that the mAP (mean Average Precision) of VGG16-SSD is 95.34%, which is 4.76% higher than MobileNet v2-SSD and 4.53% higher than EfficientNet b3-SSD. Therefore, VGG16-SSD can effectively detect wildfires in the early stages.
野火，又称森林火灾，是指通常发生在森林中且难以控制的火灾。如果能够在早期发现并抑制它(主要是烟雾和火焰)，对于减少损失具有重要意义。随着相关研究人员的关注，野火探测技术越来越先进，从传统的人工监控到传统的目标探测再到传感器探测、红外探测等。所涉及的各种检测方法仍然存在检测速度慢、精度低、易干扰和成本高等问题。本文从深度学习算法中选择了一种先进的目标检测方法SSD。以VGG16、MobileNet v2、EfficientNet b3为骨干网，构建3个独立的SSD网络。实验结果表明，VGG16-SSD的mAP (mean Average Precision)为95.34%，比MobileNet v2-SSD高4.76%，比EfficientNet b3-SSD高4.53%。因此，VGG16-SSD能够有效地对野火进行早期检测。
{"title":"Research on Image-based wildfire intelligent detection method","authors":"Qiang Yang, Fan Yu, G. Zhang, Dequan Guo, Ping Wang, Guangle Yao","doi":"10.1117/12.2667234","DOIUrl":"https://doi.org/10.1117/12.2667234","url":null,"abstract":"Wildfire, also known as forest fire, is fire that usually occur in forests and are difficult to control. If it could be detected and suppressed at an early stage (mainly smoke and flames), it has important meaning for reducing the loss. With the attention of relevant researchers, wildfire detection technology has become more and more advanced, from traditional manual monitoring to traditional target detection to sensor detection and infrared detection, etc. The various detection methods involved still have problems such as slow detection speed, low accuracy, easy interference and high cost. In this paper, SSD, an advanced target detection method, was chosen from deep learning algorithms. Three independent SSD networks are built with VGG16, MobileNet v2, and EfficientNet b3 as the backbone. The experimental results show that the mAP (mean Average Precision) of VGG16-SSD is 95.34%, which is 4.76% higher than MobileNet v2-SSD and 4.53% higher than EfficientNet b3-SSD. Therefore, VGG16-SSD can effectively detect wildfires in the early stages.","PeriodicalId":137914,"journal":{"name":"International Conference on Artificial Intelligence, Virtual Reality, and Visualization","volume":"161 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127551272","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
To solve the problem of identifying the presence of foreign objects in microfluidic chip images, an improved model is proposed for the feature of small foreign object targets. The attention mechanism is introduced to enhance the perceptiveness of the model in channel and space. The ResUnit module in the network is modified to enhance the feature information. Also choose diou as the loss function to improve the edge accuracy. The experimental results show that the improved YOLOx target detection algorithm has a significant improvement in foreign object detection in terms of accuracy, and the average precision (AP) reaches 99.12% on YOLOx, which is 0.7% higher than the original network. The results show that the improved algorithm based on YOLOx in this study can achieve foreign object detection in microfluidic chip images.
{"title":"Microfluidic chip foreign body detection based on improved YOLOx","authors":"Haodong Yan, Limin Liao, Xiaodong Liu","doi":"10.1117/12.2667312","DOIUrl":"https://doi.org/10.1117/12.2667312","url":null,"abstract":"To solve the problem of identifying the presence of foreign objects in microfluidic chip images, an improved model is proposed for the feature of small foreign object targets. The attention mechanism is introduced to enhance the perceptiveness of the model in channel and space. The ResUnit module in the network is modified to enhance the feature information. Also choose diou as the loss function to improve the edge accuracy. The experimental results show that the improved YOLOx target detection algorithm has a significant improvement in foreign object detection in terms of accuracy, and the average precision (AP) reaches 99.12% on YOLOx, which is 0.7% higher than the original network. The results show that the improved algorithm based on YOLOx in this study can achieve foreign object detection in microfluidic chip images.","PeriodicalId":137914,"journal":{"name":"International Conference on Artificial Intelligence, Virtual Reality, and Visualization","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123409585","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Along with the rapid development of the air transportation industry, the impact of aircraft wake vortices on flight safety and airport capacity has become increasingly prominent. In this paper, we propose a transformer-based model to solve the problem of multiple LIDAR wake vortex detection and recognition in airports. By setting up multiple Doppler LIDARs in the near-Earth flight areas of different runways of Shenzhen Baoan Airport (SZX), a large amount of accurate wind field data is captured for wake vortex data collection. In the deep learning framework, the radial velocity sequence obtained from the LIDAR is used as the input of the transformer. Meanwhile, local meteorological information and LIDAR operating parameters are introduced into the model, providing prior knowledge at different observation points. The experimental results show that the model has unified modeling for different LIDAR wake vortex detection, and has obtained excellent recognition results.
{"title":"Near-Earth aircraft wake vortex recognition based on multiple LIDAR and transformer","authors":"Weijun Pan, An-ning Wang","doi":"10.1117/12.2667212","DOIUrl":"https://doi.org/10.1117/12.2667212","url":null,"abstract":"Along with the rapid development of the air transportation industry, the impact of aircraft wake vortices on flight safety and airport capacity has become increasingly prominent. In this paper, we propose a transformer-based model to solve the problem of multiple LIDAR wake vortex detection and recognition in airports. By setting up multiple Doppler LIDARs in the near-Earth flight areas of different runways of Shenzhen Baoan Airport (SZX), a large amount of accurate wind field data is captured for wake vortex data collection. In the deep learning framework, the radial velocity sequence obtained from the LIDAR is used as the input of the transformer. Meanwhile, local meteorological information and LIDAR operating parameters are introduced into the model, providing prior knowledge at different observation points. The experimental results show that the model has unified modeling for different LIDAR wake vortex detection, and has obtained excellent recognition results.","PeriodicalId":137914,"journal":{"name":"International Conference on Artificial Intelligence, Virtual Reality, and Visualization","volume":"126 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114907686","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The traditional monitoring means of airport asphalt runway construction has problems such as irregular construction operation, unreliable process control and unscientific quality assessment. To meet the higher requirements of runway construction quality monitoring process, this paper analyzes the mechanized asphalt pavement construction process and the root causes affecting the construction quality, researches the application of Internet of Things (IoT) technology in asphalt pavement mechanized construction monitoring information system, and also develops the overall structure design of asphalt pavement monitoring system based on IoT technology, completes the development of a remote monitoring system based on the Web through serial communication, network protocol and database design. Finally, the system was analyzed in the test results of the west runway overhaul project of Capital International Airport. The results showed that the system has good hardware seismic resistance, good data integrity and real-time performance, and high reliability. The active and effective use of the airport asphalt runway construction management system can reflect the runway construction process comprehensively, while it is important to promote the traditional construction monitoring to advanced automated real-time process monitoring and management.
{"title":"Research on the whole process of construction quality monitoring of airport asphalt road surface based on IoT","authors":"Huanjing Jiao, Xiaobo Zhu, Kaidi Liu, Lei Guo, Kaidi Liu, Q. Ye","doi":"10.1117/12.2667210","DOIUrl":"https://doi.org/10.1117/12.2667210","url":null,"abstract":"The traditional monitoring means of airport asphalt runway construction has problems such as irregular construction operation, unreliable process control and unscientific quality assessment. To meet the higher requirements of runway construction quality monitoring process, this paper analyzes the mechanized asphalt pavement construction process and the root causes affecting the construction quality, researches the application of Internet of Things (IoT) technology in asphalt pavement mechanized construction monitoring information system, and also develops the overall structure design of asphalt pavement monitoring system based on IoT technology, completes the development of a remote monitoring system based on the Web through serial communication, network protocol and database design. Finally, the system was analyzed in the test results of the west runway overhaul project of Capital International Airport. The results showed that the system has good hardware seismic resistance, good data integrity and real-time performance, and high reliability. The active and effective use of the airport asphalt runway construction management system can reflect the runway construction process comprehensively, while it is important to promote the traditional construction monitoring to advanced automated real-time process monitoring and management.","PeriodicalId":137914,"journal":{"name":"International Conference on Artificial Intelligence, Virtual Reality, and Visualization","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121179207","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In recent years, meta-learning has become a mainstream technique for few-shot learning, and it has been widely used and achieved good results in computer vision and image processing. Based on this powerful empirical performance, we are interested in using Meta-learning frameworks in NLP to deal with the task of few-shot learning (FSL). However, due to the sparse sample size, sample-level comparisons based on other expressions are highly susceptible to interference, leading to serious overfitting problems. To achieve classification tasks, we suggest a novel Adaptive Cross-Capsule Network (ACCN) for learning generalized representations. A dynamic routing technique is utilized with the concept of a prototype network to train the support set to generalize the generalized representations of each category. The support set and the query set can fully interact dynamically to capture the essential semantic aspects of the query set following a successful non-parametric cross-attention method. Experimental results show that ACCN proposed in this paper is well adaptive to the intention classification task under additional categories, which obtain SOTA results on FewRel Datasets, which also can perform significantly better than the original classification system on Huffpost Datasets. This provides a crucial foundation for this study.
{"title":"Few shot text classification using adaptive cross capsule network","authors":"Bin Qin, Yumeng Yan, Hongyu Chen","doi":"10.1117/12.2667207","DOIUrl":"https://doi.org/10.1117/12.2667207","url":null,"abstract":"In recent years, meta-learning has become a mainstream technique for few-shot learning, and it has been widely used and achieved good results in computer vision and image processing. Based on this powerful empirical performance, we are interested in using Meta-learning frameworks in NLP to deal with the task of few-shot learning (FSL). However, due to the sparse sample size, sample-level comparisons based on other expressions are highly susceptible to interference, leading to serious overfitting problems. To achieve classification tasks, we suggest a novel Adaptive Cross-Capsule Network (ACCN) for learning generalized representations. A dynamic routing technique is utilized with the concept of a prototype network to train the support set to generalize the generalized representations of each category. The support set and the query set can fully interact dynamically to capture the essential semantic aspects of the query set following a successful non-parametric cross-attention method. Experimental results show that ACCN proposed in this paper is well adaptive to the intention classification task under additional categories, which obtain SOTA results on FewRel Datasets, which also can perform significantly better than the original classification system on Huffpost Datasets. This provides a crucial foundation for this study.","PeriodicalId":137914,"journal":{"name":"International Conference on Artificial Intelligence, Virtual Reality, and Visualization","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130099580","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Identification of traditional Chinese medicine is the core content in the practice teaching of traditional Chinese medicine. It requires students to master the identification method of traditional Chinese medicine and have the ability of clinical application. In daily teaching, due to the large loss of Chinese medicinal materials, the shortage of precious medicinal materials, the lack of living medicinal materials and the long observation period at each stage of living materials, the teaching effect is not good, which affects the improvement of students' ability to identify Chinese medicinal materials. Three-dimensional teaching resources can carry out three-dimensional simulation of the growth process of medicinal plants of Traditional Chinese medicine, Chinese medicine decoction pieces and their medicinal plants, help students build knowledge and improve their identification ability of medicinal materials. This paper summarizes the advantages of three-dimensional teaching resources. The development process of 3D teaching resources of Chinese medicinal materials was elaborated in detail. The development method and implementation process of 3D modeling, 3D animation, construction of virtual scene of medicinal herbs growing environment and interactive roaming of scene are emphasized. The optimization methods of model, animation and scene are discussed.
{"title":"Development of three-dimensional dynamic teaching resources of traditional Chinese medicine","authors":"Li Wang","doi":"10.1117/12.2667870","DOIUrl":"https://doi.org/10.1117/12.2667870","url":null,"abstract":"Identification of traditional Chinese medicine is the core content in the practice teaching of traditional Chinese medicine. It requires students to master the identification method of traditional Chinese medicine and have the ability of clinical application. In daily teaching, due to the large loss of Chinese medicinal materials, the shortage of precious medicinal materials, the lack of living medicinal materials and the long observation period at each stage of living materials, the teaching effect is not good, which affects the improvement of students' ability to identify Chinese medicinal materials. Three-dimensional teaching resources can carry out three-dimensional simulation of the growth process of medicinal plants of Traditional Chinese medicine, Chinese medicine decoction pieces and their medicinal plants, help students build knowledge and improve their identification ability of medicinal materials. This paper summarizes the advantages of three-dimensional teaching resources. The development process of 3D teaching resources of Chinese medicinal materials was elaborated in detail. The development method and implementation process of 3D modeling, 3D animation, construction of virtual scene of medicinal herbs growing environment and interactive roaming of scene are emphasized. The optimization methods of model, animation and scene are discussed.","PeriodicalId":137914,"journal":{"name":"International Conference on Artificial Intelligence, Virtual Reality, and Visualization","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130478459","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The health management and maintenance of photovoltaic (PV) plants are inherent problems in the PV industry. The need to establish digital positioning for each PV string and PV module is urgent. This paper provides a complete end-to-end system for digital segmentation and localization of PV strings and modules on the aerial orthophotos. The system includes three main parts: (1) the dataset built from the images captured by Unmanned Aerial Vehicles (UAV) and corresponding image preprocessing techniques. (2) a modified Deeplab V3+ neural network is designed to extract the PV strings in the aerial orthophotos. (3) a PV module extraction algorithm is introduced to get the centroid of every PV module and the sliding window strategy is adopted to avoid the chopped PV strings problem. With the above process, the digital location information of PV panels can be correlated with the actual physical information. We conduct detailed experiments with actual scene data and different models. The extensive results confirm the accuracy and efficiency of the system proposed in this paper with comparative analysis.
{"title":"Deeplab V3+ based segmentation method for PV panels with aerial orthoimages","authors":"Junwen Wang, Min Liu, Wenjun Yan","doi":"10.1117/12.2667262","DOIUrl":"https://doi.org/10.1117/12.2667262","url":null,"abstract":"The health management and maintenance of photovoltaic (PV) plants are inherent problems in the PV industry. The need to establish digital positioning for each PV string and PV module is urgent. This paper provides a complete end-to-end system for digital segmentation and localization of PV strings and modules on the aerial orthophotos. The system includes three main parts: (1) the dataset built from the images captured by Unmanned Aerial Vehicles (UAV) and corresponding image preprocessing techniques. (2) a modified Deeplab V3+ neural network is designed to extract the PV strings in the aerial orthophotos. (3) a PV module extraction algorithm is introduced to get the centroid of every PV module and the sliding window strategy is adopted to avoid the chopped PV strings problem. With the above process, the digital location information of PV panels can be correlated with the actual physical information. We conduct detailed experiments with actual scene data and different models. The extensive results confirm the accuracy and efficiency of the system proposed in this paper with comparative analysis.","PeriodicalId":137914,"journal":{"name":"International Conference on Artificial Intelligence, Virtual Reality, and Visualization","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133937375","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
As the most intuitive and reliable monitoring quantity of concrete dams, deformation can comprehensively reflect the service performance of dams in real time. By constructing a real-time prediction model, it has important guiding significance for the identification and response of deformation anomalies in the operation of water conservancy projects. In this paper, a deep learning algorithm: long-term and short-term memory neural network (LSTM), combined with attention mechanism, is used to construct the deformation prediction model of concrete dam. Through engineering examples, the MSE of LSTM model with attention mechanism is 0.69, and the MAE is 0.67. Compared with the stepwise regression model, the recurrent neural network model (RNN) and the LSTM model without attention mechanism, the errors are reduced. LSTM can better mine the long-term and short-term dependencies in deformation sequences, and use the attention mechanism to influence the global and local relationships between factors, highlighting the contribution of main factors to deformation.
{"title":"Concrete dam deformation prediction method based on improved LSTM deep learning","authors":"W. Li, Yifan Tan, Li-fu Xu","doi":"10.1117/12.2667216","DOIUrl":"https://doi.org/10.1117/12.2667216","url":null,"abstract":"As the most intuitive and reliable monitoring quantity of concrete dams, deformation can comprehensively reflect the service performance of dams in real time. By constructing a real-time prediction model, it has important guiding significance for the identification and response of deformation anomalies in the operation of water conservancy projects. In this paper, a deep learning algorithm: long-term and short-term memory neural network (LSTM), combined with attention mechanism, is used to construct the deformation prediction model of concrete dam. Through engineering examples, the MSE of LSTM model with attention mechanism is 0.69, and the MAE is 0.67. Compared with the stepwise regression model, the recurrent neural network model (RNN) and the LSTM model without attention mechanism, the errors are reduced. LSTM can better mine the long-term and short-term dependencies in deformation sequences, and use the attention mechanism to influence the global and local relationships between factors, highlighting the contribution of main factors to deformation.","PeriodicalId":137914,"journal":{"name":"International Conference on Artificial Intelligence, Virtual Reality, and Visualization","volume":"211 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116408522","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
How to achieve effective exploration is a key issue in the training of Reinforcement learning. The known exploration policy addresses this issue by adding noise to the policy for guiding the agent exploring. However, it has two problems that 1) the exploration scale has low adaptability to the training stability due to the added noise from a fixed distribution and 2) the policy learned after the training may be locally optimal because the exploration is insufficient. Adaptive exploration policy addresses the first problem by adjusting the noise scale according to the training stability. But the learned policy may still be locally optimal. In this paper, we propose an adaptive exploration network policy to address this problem by considering exploration direction. The motivation is that the agent should explore in the direction of increasing the sample diversity to avoid the local optimum caused by insufficient exploration. Firstly, we construct a prediction network to predict the next state after the agent makes a decision at the current state. Secondly, we propose an exploration network to generate the exploration direction. To increase the sample diversity, this network is trained by maximizing the distance between the predicted next state from prediction network and the current state. Then we adjust the exploration scale to adapt to the training stability. Finally, we propose adaptive exploration network policy based on the new noise constructed by the generated exploration direction and the adaptive exploration scale. Experiments illustrate the effectiveness of our method.
{"title":"Adaptive exploration network policy for effective exploration in reinforcement learning","authors":"Min Li, William Zhu","doi":"10.1117/12.2667206","DOIUrl":"https://doi.org/10.1117/12.2667206","url":null,"abstract":"How to achieve effective exploration is a key issue in the training of Reinforcement learning. The known exploration policy addresses this issue by adding noise to the policy for guiding the agent exploring. However, it has two problems that 1) the exploration scale has low adaptability to the training stability due to the added noise from a fixed distribution and 2) the policy learned after the training may be locally optimal because the exploration is insufficient. Adaptive exploration policy addresses the first problem by adjusting the noise scale according to the training stability. But the learned policy may still be locally optimal. In this paper, we propose an adaptive exploration network policy to address this problem by considering exploration direction. The motivation is that the agent should explore in the direction of increasing the sample diversity to avoid the local optimum caused by insufficient exploration. Firstly, we construct a prediction network to predict the next state after the agent makes a decision at the current state. Secondly, we propose an exploration network to generate the exploration direction. To increase the sample diversity, this network is trained by maximizing the distance between the predicted next state from prediction network and the current state. Then we adjust the exploration scale to adapt to the training stability. Finally, we propose adaptive exploration network policy based on the new noise constructed by the generated exploration direction and the adaptive exploration scale. Experiments illustrate the effectiveness of our method.","PeriodicalId":137914,"journal":{"name":"International Conference on Artificial Intelligence, Virtual Reality, and Visualization","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123621611","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：