Pub Date : 2025-08-11DOI: 10.1007/s43684-025-00101-4
Yuyang Li, Minghui Liwang, Li Li
Machine learning, a revolutionary and advanced technology, has been widely applied in the field of stock trading. However, training an autonomous trading strategy which can effectively balance risk and Return On Investment without human supervision in the stock market with high uncertainty is still a bottleneck. This paper constructs a Bayesian-inferenced Gated Recurrent Unit architecture to support long-term stock price prediction based on characteristics of the stock information learned from historical data, augmented with memory of recent up- and-down fluctuations occur in the data of short-term stock movement. The Gated Recurrent Unit architecture incorporates uncertainty estimation into the prediction process, which take care of decision-making in an ever-changing dynamic environment. Three trading strategies were implemented in this model; namely, a Price Model Strategy, a Probabilistic Model Strategy, and a Bayesian Gated Recurrent Unit Strategy, each leveraging the respective model’s outputs to optimize trading decisions. The experimental results show that, compared with the standard Gated Recurrent Unit models, the modified model exhibits a huge tremendous/dramatic advantage in managing volatility and improving return on investment Return On Investment. The results and findings underscore the significant potential of combining Bayesian inference with machine learning to operate effectively in chaotic decision-making environments.
{"title":"Learning to trade autonomously in stocks and shares: integrating uncertainty into trading strategies","authors":"Yuyang Li, Minghui Liwang, Li Li","doi":"10.1007/s43684-025-00101-4","DOIUrl":"10.1007/s43684-025-00101-4","url":null,"abstract":"<div><p>Machine learning, a revolutionary and advanced technology, has been widely applied in the field of stock trading. However, training an autonomous trading strategy which can effectively balance risk and Return On Investment without human supervision in the stock market with high uncertainty is still a bottleneck. This paper constructs a Bayesian-inferenced Gated Recurrent Unit architecture to support long-term stock price prediction based on characteristics of the stock information learned from historical data, augmented with memory of recent up- and-down fluctuations occur in the data of short-term stock movement. The Gated Recurrent Unit architecture incorporates uncertainty estimation into the prediction process, which take care of decision-making in an ever-changing dynamic environment. Three trading strategies were implemented in this model; namely, a Price Model Strategy, a Probabilistic Model Strategy, and a Bayesian Gated Recurrent Unit Strategy, each leveraging the respective model’s outputs to optimize trading decisions. The experimental results show that, compared with the standard Gated Recurrent Unit models, the modified model exhibits a huge tremendous/dramatic advantage in managing volatility and improving return on investment Return On Investment. The results and findings underscore the significant potential of combining Bayesian inference with machine learning to operate effectively in chaotic decision-making environments.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-025-00101-4.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144810783","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-01DOI: 10.1016/j.vrih.2022.08.014
Xinrong Hu, Kaifan Yang, Ruiqi Luo, Tao Peng, Junping Liu
With the popularity of the digital human body, monocular three-dimensional (3D) face reconstruction is widely used in fields such as animation and face recognition. Although current methods trained using single-view image sets perform well in monocular 3D face reconstruction tasks, they tend to rely on the constraints of the a priori model or the appearance conditions of the input images, fundamentally because of the inability to propose an effective method to reduce the effects of two-dimensional (2D) ambiguity. To solve this problem, we developed an unsupervised training framework for monocular face 3D reconstruction using rotational cycle consistency. Specifically, to learn more accurate facial information, we first used an autoencoder to factor the input images and applied these factors to generate normalized frontal views. We then proceeded through a differentiable renderer to use rotational consistency to continuously perceive refinement. Our method provided implicit multi-view consistency constraints on the pose and depth information estimation of the input face, and the performance was accurate and robust in the presence of large variations in expression and pose. In the benchmark tests, our method performed more stably and realistically than other methods that used 3D face reconstruction in monocular 2D images.
{"title":"Learning monocular face reconstruction from in the wild images using rotation cycle consistency","authors":"Xinrong Hu, Kaifan Yang, Ruiqi Luo, Tao Peng, Junping Liu","doi":"10.1016/j.vrih.2022.08.014","DOIUrl":"10.1016/j.vrih.2022.08.014","url":null,"abstract":"<div><div>With the popularity of the digital human body, monocular three-dimensional (3D) face reconstruction is widely used in fields such as animation and face recognition. Although current methods trained using single-view image sets perform well in monocular 3D face reconstruction tasks, they tend to rely on the constraints of the a priori model or the appearance conditions of the input images, fundamentally because of the inability to propose an effective method to reduce the effects of two-dimensional (2D) ambiguity. To solve this problem, we developed an unsupervised training framework for monocular face 3D reconstruction using rotational cycle consistency. Specifically, to learn more accurate facial information, we first used an autoencoder to factor the input images and applied these factors to generate normalized frontal views. We then proceeded through a differentiable renderer to use rotational consistency to continuously perceive refinement. Our method provided implicit multi-view consistency constraints on the pose and depth information estimation of the input face, and the performance was accurate and robust in the presence of large variations in expression and pose. In the benchmark tests, our method performed more stably and realistically than other methods that used 3D face reconstruction in monocular 2D images.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"7 4","pages":"Pages 379-392"},"PeriodicalIF":0.0,"publicationDate":"2025-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144903888","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-01DOI: 10.1016/j.vrih.2023.10.006
Rui Song , Xiaoying Sun , Dangxiao Wang , Guohong Liu , Dongyan Nie
High-fidelity tactile rendering offers significant potential for improving the richness and immersion of touchscreen interactions. This study focuses on a quantitative description of tactile rendering fidelity using a custom-designed hybrid electrovibration and mechanical vibration (HEM) device. An electrovibration and mechanical vibration (EMV) algorithm that renders 3D gratings with different physical heights was proposed and shown to achieve 81% accuracy in shape recognition. Models of tactile rendering fidelity were established based on the evaluation of the height discrimination threshold, and the psychophysical-physical relationships between the discrimination and reference heights were well described by a modification of Weber’s law, with correlation coefficients higher than 0.9. The physiological-physical relationship between the pulse firing rate and the physical stimulation voltage was modeled using the Izhikevich spiking model with a logarithmic relationship.
{"title":"Psychological and physiological model of tactile rendering fidelity using combined electro and mechanical vibration","authors":"Rui Song , Xiaoying Sun , Dangxiao Wang , Guohong Liu , Dongyan Nie","doi":"10.1016/j.vrih.2023.10.006","DOIUrl":"10.1016/j.vrih.2023.10.006","url":null,"abstract":"<div><div>High-fidelity tactile rendering offers significant potential for improving the richness and immersion of touchscreen interactions. This study focuses on a quantitative description of tactile rendering fidelity using a custom-designed hybrid electrovibration and mechanical vibration (HEM) device. An electrovibration and mechanical vibration (EMV) algorithm that renders 3D gratings with different physical heights was proposed and shown to achieve 81% accuracy in shape recognition. Models of tactile rendering fidelity were established based on the evaluation of the height discrimination threshold, and the psychophysical-physical relationships between the discrimination and reference heights were well described by a modification of Weber’s law, with correlation coefficients higher than 0.9. The physiological-physical relationship between the pulse firing rate and the physical stimulation voltage was modeled using the Izhikevich spiking model with a logarithmic relationship.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"7 4","pages":"Pages 344-366"},"PeriodicalIF":0.0,"publicationDate":"2025-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144904024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-01DOI: 10.1016/j.vrih.2025.05.001
Ruicheng Gao , Yue Qi
Background
Physics-based differentiable rendering (PBDR) aims to propagate gradients from scene parameters to image pixels or vice versa. The physically correct gradients obtained can be used in various applications, including inverse rendering and machine learning. Currently, two categories of methods are prevalent in the PBDR community: reparameterization and boundary sampling methods. The state-of-the-art boundary sampling methods rely on a guiding structure to calculate the gradients efficiently. They utilize the rays generated in traditional path-tracing methods and project them onto the object silhouette boundary to initialize the guiding structure.
Methods
In this study, we propose an augmentation of previous projective-sampling-based boundary-sampling methods in a bidirectional manner. Specifically, we utilize the rays spawned from the sensors and also employ the rays emitted by the emitters to initialize the guiding structure.
Results
To demonstrate the benefits of our technique, we perform a comparative analysis of differentiable rendering and inverse rendering performance. We utilize a range of synthetic scene examples and evaluate our method against state-of-the-art projective-sampling-based differentiable rendering methods.
Conclusions
The experiments show that our method achieves lower variance gradients in the forward differentiable rendering process and better geometry reconstruction quality in the inverse-rendering results.
{"title":"Bidirectional projective sampling for physics-based differentiable rendering","authors":"Ruicheng Gao , Yue Qi","doi":"10.1016/j.vrih.2025.05.001","DOIUrl":"10.1016/j.vrih.2025.05.001","url":null,"abstract":"<div><h3>Background</h3><div>Physics-based differentiable rendering (PBDR) aims to propagate gradients from scene parameters to image pixels or vice versa. The physically correct gradients obtained can be used in various applications, including inverse rendering and machine learning. Currently, two categories of methods are prevalent in the PBDR community: reparameterization and boundary sampling methods. The state-of-the-art boundary sampling methods rely on a guiding structure to calculate the gradients efficiently. They utilize the rays generated in traditional path-tracing methods and project them onto the object silhouette boundary to initialize the guiding structure.</div></div><div><h3>Methods</h3><div>In this study, we propose an augmentation of previous projective-sampling-based boundary-sampling methods in a bidirectional manner. Specifically, we utilize the rays spawned from the sensors and also employ the rays emitted by the emitters to initialize the guiding structure.</div></div><div><h3>Results</h3><div>To demonstrate the benefits of our technique, we perform a comparative analysis of differentiable rendering and inverse rendering performance. We utilize a range of synthetic scene examples and evaluate our method against state-of-the-art projective-sampling-based differentiable rendering methods.</div></div><div><h3>Conclusions</h3><div>The experiments show that our method achieves lower variance gradients in the forward differentiable rendering process and better geometry reconstruction quality in the inverse-rendering results.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"7 4","pages":"Pages 367-378"},"PeriodicalIF":0.0,"publicationDate":"2025-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144903887","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-01DOI: 10.1016/j.vrih.2023.08.004
Biao Dong , Wenjun Tan , Weichao Chang , Baoting Li , Yanliang Guo , Quanxing Hu , Guangwei Liu
Background
As information technology has advanced and been popularized, open pit mining has rapidly developed toward integration and digitization. The three-dimensional reconstruction technology has been successfully applied to geological reconstruction and modeling of surface scenes in open pit mines. However, an integrated modeling method for surface and underground mine sites has not been reported.
Methods
In this study, we propose an integrated modeling method for open pit mines that fuses a real scene on the surface with an underground geological model. Based on oblique photography, a real-scene model was established on the surface. Based on the surface-stitching method proposed, the upper and lower surfaces and sides of the model were constructed in stages to construct a complete underground three-dimensional geological model, and the aboveground and underground models were registered together to build an integrated open pit mine model.
Results
The oblique photography method used reconstructed a surface model of an open pit mine using a real scene. The surface-stitching algorithm proposed was compared with the ball-pivoting and Poisson algorithms, and the integrity of the reconstructed model was markedly superior to that of the other two reconstruction methods. In addition, the surface-stitching algorithm was applied to the reconstruction of different formation models and showed good stability and reconstruction efficiency. Finally, the aboveground and underground models were accurately fitted after registration to form an integrated model.
Conclusions
The proposed method can efficiently establish an integrated open pit model. Based on the integrated model, an open pit auxiliary planning system was designed and realized. It supports the functions of mining planning and output calculation, assists users in mining planning and operation management, and improves production efficiency and management levels.
{"title":"Integrating models of real aboveground scene and underground geological structures at an open pit mine","authors":"Biao Dong , Wenjun Tan , Weichao Chang , Baoting Li , Yanliang Guo , Quanxing Hu , Guangwei Liu","doi":"10.1016/j.vrih.2023.08.004","DOIUrl":"10.1016/j.vrih.2023.08.004","url":null,"abstract":"<div><h3>Background</h3><div>As information technology has advanced and been popularized, open pit mining has rapidly developed toward integration and digitization. The three-dimensional reconstruction technology has been successfully applied to geological reconstruction and modeling of surface scenes in open pit mines. However, an integrated modeling method for surface and underground mine sites has not been reported.</div></div><div><h3>Methods</h3><div>In this study, we propose an integrated modeling method for open pit mines that fuses a real scene on the surface with an underground geological model. Based on oblique photography, a real-scene model was established on the surface. Based on the surface-stitching method proposed, the upper and lower surfaces and sides of the model were constructed in stages to construct a complete underground three-dimensional geological model, and the aboveground and underground models were registered together to build an integrated open pit mine model.</div></div><div><h3>Results</h3><div>The oblique photography method used reconstructed a surface model of an open pit mine using a real scene. The surface-stitching algorithm proposed was compared with the ball-pivoting and Poisson algorithms, and the integrity of the reconstructed model was markedly superior to that of the other two reconstruction methods. In addition, the surface-stitching algorithm was applied to the reconstruction of different formation models and showed good stability and reconstruction efficiency. Finally, the aboveground and underground models were accurately fitted after registration to form an integrated model.</div></div><div><h3>Conclusions</h3><div>The proposed method can efficiently establish an integrated open pit model. Based on the integrated model, an open pit auxiliary planning system was designed and realized. It supports the functions of mining planning and output calculation, assists users in mining planning and operation management, and improves production efficiency and management levels.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"7 4","pages":"Pages 406-420"},"PeriodicalIF":0.0,"publicationDate":"2025-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144903890","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-01DOI: 10.1016/j.vrih.2024.08.004
Ehsan Shourangiz, Fatemeh Ghafari, Chao Wang
The integration of Human-Robot Collaboration (HRC) into Virtual Reality (VR) technology is transforming industries by enhancing workforce skills, improving safety, and optimizing operational processes and efficiency through realistic simulations of industry-specific scenarios. Despite the growing adoption of VR integrated with HRC, comprehensive reviews of current research in HRC-VR within the construction and manufacturing fields are lacking. This review examines the latest advances in designing and implementing HRC using VR technology in these industries. The aim is to address the application domains of HRC-VR, types of robots used, VR setups, and software solutions used. To achieve this, a systematic literature review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses methodology was conducted on the Web of Science and Google Scholar databases, analyzing 383 articles and selecting 53 papers that met the established selection criteria. The findings emphasize a significant focus on enhancing human-robot interaction with a trend toward using immersive VR experiences and interactive 3D content creation tools. However, the integration of HRC with VR, especially in the dynamic construction environment, presents unique challenges and opportunities for future research, including developing more realistic simulations and adaptable robot systems. This paper offers insights for researchers, practitioners, educators, industry professionals, and policymakers interested in leveraging the integration of HRC with VR in construction and manufacturing industries.
将人机协作（HRC）集成到虚拟现实（VR）技术中，通过对行业特定场景的逼真模拟，提高劳动力技能、提高安全性、优化操作流程和效率，正在改变行业。尽管越来越多地采用VR与HRC相结合的技术，但目前在建筑和制造领域对HRC-VR的研究还缺乏全面的综述。本文综述了在这些行业中使用VR技术设计和实施HRC的最新进展。目的是解决HRC-VR的应用领域，使用的机器人类型，VR设置和使用的软件解决方案。为了实现这一目标，我们在Web of Science和b谷歌Scholar数据库上使用首选报告项目进行了系统文献综述和meta分析方法，分析了383篇文章，并选择了53篇符合既定选择标准的论文。研究结果强调，通过使用沉浸式VR体验和交互式3D内容创作工具的趋势，增强人机交互是一个重要的重点。然而，HRC与VR的融合，特别是在动态建筑环境中，为未来的研究带来了独特的挑战和机遇，包括开发更逼真的模拟和适应性强的机器人系统。本文为研究人员、从业人员、教育工作者、行业专业人士和政策制定者提供了见解，他们对在建筑和制造业中利用HRC与VR的集成感兴趣。
{"title":"Human-robot collaboration integrated with virtual reality in construction and manufacturing industries: A systematic review","authors":"Ehsan Shourangiz, Fatemeh Ghafari, Chao Wang","doi":"10.1016/j.vrih.2024.08.004","DOIUrl":"10.1016/j.vrih.2024.08.004","url":null,"abstract":"<div><div>The integration of Human-Robot Collaboration (HRC) into Virtual Reality (VR) technology is transforming industries by enhancing workforce skills, improving safety, and optimizing operational processes and efficiency through realistic simulations of industry-specific scenarios. Despite the growing adoption of VR integrated with HRC, comprehensive reviews of current research in HRC-VR within the construction and manufacturing fields are lacking. This review examines the latest advances in designing and implementing HRC using VR technology in these industries. The aim is to address the application domains of HRC-VR, types of robots used, VR setups, and software solutions used. To achieve this, a systematic literature review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses methodology was conducted on the Web of Science and Google Scholar databases, analyzing 383 articles and selecting 53 papers that met the established selection criteria. The findings emphasize a significant focus on enhancing human-robot interaction with a trend toward using immersive VR experiences and interactive 3D content creation tools. However, the integration of HRC with VR, especially in the dynamic construction environment, presents unique challenges and opportunities for future research, including developing more realistic simulations and adaptable robot systems. This paper offers insights for researchers, practitioners, educators, industry professionals, and policymakers interested in leveraging the integration of HRC with VR in construction and manufacturing industries.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"7 4","pages":"Pages 317-343"},"PeriodicalIF":0.0,"publicationDate":"2025-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144903886","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-01DOI: 10.1016/j.vrih.2022.08.013
Erwan Leria, Markku Makitalo, Julius Ikkala, Pekka Jääskeläinen
Stereoscopic and multiview rendering are used for virtual reality and the synthetic generation of light fields from three-dimensional scenes. Because rendering multiple views using ray tracing techniques is computationally expensive, the utilization of multiprocessor machines is necessary to achieve real-time frame rates. In this study, we propose a dynamic load-balancing algorithm for real-time multiview path tracing on multi-compute device platforms. The proposed algorithm was adapted to heterogeneous hardware combinations and dynamic scenes in real time. We show that on a heterogeneous dual-GPU platform, our implementation reduces the rendering time by an average of approximately 30%–50% compared with that of a uniform workload distribution, depending on the scene and number of views.
{"title":"Dynamic load balancing for real-time multiview path tracing on multi-GPU architectures","authors":"Erwan Leria, Markku Makitalo, Julius Ikkala, Pekka Jääskeläinen","doi":"10.1016/j.vrih.2022.08.013","DOIUrl":"10.1016/j.vrih.2022.08.013","url":null,"abstract":"<div><div>Stereoscopic and multiview rendering are used for virtual reality and the synthetic generation of light fields from three-dimensional scenes. Because rendering multiple views using ray tracing techniques is computationally expensive, the utilization of multiprocessor machines is necessary to achieve real-time frame rates. In this study, we propose a dynamic load-balancing algorithm for real-time multiview path tracing on multi-compute device platforms. The proposed algorithm was adapted to heterogeneous hardware combinations and dynamic scenes in real time. We show that on a heterogeneous dual-GPU platform, our implementation reduces the rendering time by an average of approximately 30%–50% compared with that of a uniform workload distribution, depending on the scene and number of views.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"7 4","pages":"Pages 393-405"},"PeriodicalIF":0.0,"publicationDate":"2025-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144903889","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-07-29DOI: 10.1007/s43684-025-00103-2
André Luiz Carvalho Ottoni
AutoML systems seek to assist Artificial Intelligence users in finding the best configurations for machine learning models. Following this line, recently the area of Automated Reinforcement Learning (AutoRL) has become increasingly relevant, given the growing increase in applications for reinforcement learning algorithms. However, the literature still lacks specific AutoRL systems for combinatorial optimization, especially for the Sequential Ordering Problem (SOP). Therefore, this paper aims to present a new AutoRL approach for SOP. For this, two new methods are proposed using hyperparameter optimization and metalearning: AutoRL-SOP and AutoRL-SOP-MtL. The proposed AutoRL techniques enable the combined tuning of three SARSA hyperparameters, being ϵ-greedy policy, learning rate, and discount factor. Furthermore, the new metalearning approach enables the transfer of hyperparameters between two combinatorial optimization domains: TSP (source) and SOP (target). The results show that the application of metalearning generates a reduction in computational cost in hyperparameter optimization. Furthermore, the proposed AutoRL methods achieved the best solutions in 23 out of 28 simulated TSPLIB instances compared to recent literature studies.
{"title":"Automated reinforcement learning for sequential ordering problem using hyperparameter optimization and metalearning","authors":"André Luiz Carvalho Ottoni","doi":"10.1007/s43684-025-00103-2","DOIUrl":"10.1007/s43684-025-00103-2","url":null,"abstract":"<div><p>AutoML systems seek to assist Artificial Intelligence users in finding the best configurations for machine learning models. Following this line, recently the area of Automated Reinforcement Learning (AutoRL) has become increasingly relevant, given the growing increase in applications for reinforcement learning algorithms. However, the literature still lacks specific AutoRL systems for combinatorial optimization, especially for the Sequential Ordering Problem (SOP). Therefore, this paper aims to present a new AutoRL approach for SOP. For this, two new methods are proposed using hyperparameter optimization and metalearning: AutoRL-SOP and AutoRL-SOP-MtL. The proposed AutoRL techniques enable the combined tuning of three SARSA hyperparameters, being <i>ϵ</i>-greedy policy, learning rate, and discount factor. Furthermore, the new metalearning approach enables the transfer of hyperparameters between two combinatorial optimization domains: TSP (source) and SOP (target). The results show that the application of metalearning generates a reduction in computational cost in hyperparameter optimization. Furthermore, the proposed AutoRL methods achieved the best solutions in 23 out of 28 simulated TSPLIB instances compared to recent literature studies.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-025-00103-2.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145171555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Autonomous driving systems (ADS) are at the forefront of technological innovation, promising enhanced safety, efficiency, and convenience in transportation. This study investigates the potential of end-to-end reinforcement learning (RL) architectures for ADS, specifically focusing on a Go-To-Point task involving lane-keeping and navigation through basic urban environments. The study uses the Proximal Policy Optimization (PPO) algorithm within the CARLA simulation environment. Traditional modular systems, which separate driving tasks into perception, decision-making, and control, provide interpretability and reliability in controlled scenarios but struggle with adaptability to dynamic, real-world conditions. In contrast, end-to-end systems offer a more integrated approach, potentially enhancing flexibility and decision-making cohesion.
This research introduces CARLA-GymDrive, a novel framework integrating the CARLA simulator with the Gymnasium API, enabling seamless RL experimentation with both discrete and continuous action spaces. Through a two-phase training regimen, the study evaluates the efficacy of PPO in an end-to-end ADS focused on basic tasks like lane-keeping and waypoint navigation. A comparative analysis with modular architectures is also provided. The findings highlight the strengths of PPO in managing continuous control tasks, achieving smoother and more adaptable driving behaviors than value-based algorithms like Deep Q-Networks. However, challenges remain in generalization and computational demands, with end-to-end systems requiring extensive training time.
While the study underscores the potential of end-to-end architectures, it also identifies limitations in scalability and real-world applicability, suggesting that modular systems may currently be more feasible for practical ADS deployment. Nonetheless, the CARLA-GymDrive framework and the insights gained from PPO-based ADS contribute significantly to the field, laying a foundation for future advancements in AD.
{"title":"Evaluating end-to-end autonomous driving architectures: a proximal policy optimization approach in simulated environments","authors":"Ângelo Morgado, Kaoru Ota, Mianxiong Dong, Nuno Pombo","doi":"10.1007/s43684-025-00102-3","DOIUrl":"10.1007/s43684-025-00102-3","url":null,"abstract":"<div><p>Autonomous driving systems (ADS) are at the forefront of technological innovation, promising enhanced safety, efficiency, and convenience in transportation. This study investigates the potential of end-to-end reinforcement learning (RL) architectures for ADS, specifically focusing on a Go-To-Point task involving lane-keeping and navigation through basic urban environments. The study uses the Proximal Policy Optimization (PPO) algorithm within the CARLA simulation environment. Traditional modular systems, which separate driving tasks into perception, decision-making, and control, provide interpretability and reliability in controlled scenarios but struggle with adaptability to dynamic, real-world conditions. In contrast, end-to-end systems offer a more integrated approach, potentially enhancing flexibility and decision-making cohesion.</p><p>This research introduces CARLA-GymDrive, a novel framework integrating the CARLA simulator with the Gymnasium API, enabling seamless RL experimentation with both discrete and continuous action spaces. Through a two-phase training regimen, the study evaluates the efficacy of PPO in an end-to-end ADS focused on basic tasks like lane-keeping and waypoint navigation. A comparative analysis with modular architectures is also provided. The findings highlight the strengths of PPO in managing continuous control tasks, achieving smoother and more adaptable driving behaviors than value-based algorithms like Deep Q-Networks. However, challenges remain in generalization and computational demands, with end-to-end systems requiring extensive training time.</p><p>While the study underscores the potential of end-to-end architectures, it also identifies limitations in scalability and real-world applicability, suggesting that modular systems may currently be more feasible for practical ADS deployment. Nonetheless, the CARLA-GymDrive framework and the insights gained from PPO-based ADS contribute significantly to the field, laying a foundation for future advancements in AD.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-025-00102-3.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145169256","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-20DOI: 10.1016/j.bcra.2025.100319
Rahma Mukta , Shantanu Pal , Kowshik Chowdhury , Michael Hitchens , Hye-young Paik , Salil S. Kanhere
As digital ecosystems become more complex with decentralized technologies like the Internet of Things (IoT) and blockchain, traditional access control models fail to meet the security needs of dynamic, high-risk environments. The need for dynamic, fine-grained access control mechanisms has become critical, particularly in environments where trust must be continuously evaluated, and access decisions must adapt to real-time conditions. Traditional models often rely on static identity management and centralized trust assumptions, which are inadequate for modern, decentralized, and highly dynamic environments such as IoT ecosystems. Consequently, existing solutions lack fine-grained identity management, flexible delegation, and continuous trust evaluation, highlighting the need for a more robust, adaptive, and decentralized access control architecture. To address these gaps, this paper presents a novel access control architecture that integrates self-sovereign identity (SSI) and decentralized identifier (DID)-based access control with zero trust principles, enhanced by a flexible capability-based access control (CapBAC) approach. Leveraging SSI and DID allows entities to manage their identities without relying on a central authority, aligning with zero-trust principles. The integration of CapBAC ensures flexible, context-aware, and attribute-based access control, where access rights are dynamically granted based on the requester's capabilities. This enables fine-grained delegation of access rights, allowing trusted entities to delegate specific privileges to others without compromising overall security. Continuous trust evaluation is employed to assess the authenticity of access requests, mitigating the risks posed by compromised devices or users. The proposed architecture also incorporates blockchain technology to ensure transparent, immutable, and secure management of access logs, providing traceability and accountability for all access events. We demonstrate the feasibility and effectiveness of this solution through performance evaluations and comparisons with existing access control schemes, showing its superior security, scalability, and adaptability in real-world scenarios. Our work demonstrates a comprehensive, decentralized, and scalable solution for secure access control delegation using zero trust-driven principles.
{"title":"Zero trust-driven access control delegation using blockchain","authors":"Rahma Mukta , Shantanu Pal , Kowshik Chowdhury , Michael Hitchens , Hye-young Paik , Salil S. Kanhere","doi":"10.1016/j.bcra.2025.100319","DOIUrl":"10.1016/j.bcra.2025.100319","url":null,"abstract":"<div><div>As digital ecosystems become more complex with decentralized technologies like the Internet of Things (IoT) and blockchain, traditional access control models fail to meet the security needs of dynamic, high-risk environments. The need for dynamic, fine-grained access control mechanisms has become critical, particularly in environments where trust must be continuously evaluated, and access decisions must adapt to real-time conditions. Traditional models often rely on static identity management and centralized trust assumptions, which are inadequate for modern, decentralized, and highly dynamic environments such as IoT ecosystems. Consequently, existing solutions lack fine-grained identity management, flexible delegation, and continuous trust evaluation, highlighting the need for a more robust, adaptive, and decentralized access control architecture. To address these gaps, this paper presents a novel access control architecture that integrates self-sovereign identity (SSI) and decentralized identifier (DID)-based access control with zero trust principles, enhanced by a flexible capability-based access control (CapBAC) approach. Leveraging SSI and DID allows entities to manage their identities without relying on a central authority, aligning with zero-trust principles. The integration of CapBAC ensures flexible, context-aware, and attribute-based access control, where access rights are dynamically granted based on the requester's capabilities. This enables fine-grained delegation of access rights, allowing trusted entities to delegate specific privileges to others without compromising overall security. Continuous trust evaluation is employed to assess the authenticity of access requests, mitigating the risks posed by compromised devices or users. The proposed architecture also incorporates blockchain technology to ensure transparent, immutable, and secure management of access logs, providing traceability and accountability for all access events. We demonstrate the feasibility and effectiveness of this solution through performance evaluations and comparisons with existing access control schemes, showing its superior security, scalability, and adaptability in real-world scenarios. Our work demonstrates a comprehensive, decentralized, and scalable solution for secure access control delegation using zero trust-driven principles.</div></div>","PeriodicalId":53141,"journal":{"name":"Blockchain-Research and Applications","volume":"7 1","pages":"Article 100319"},"PeriodicalIF":5.6,"publicationDate":"2025-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146024719","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号