Walking stability is one of the key issues for humanoid robots. A self-stabilised walking gait for a full dynamic model of humanoid robots is proposed. For simplified models, that is, the linear inverted pendulum model and variable-length inverted pendulum model, self-stabilisation of walking gait can be obtained if virtual constraints are properly defined. This result is extended to the full dynamic model of humanoid robots by using an essential dynamic model, which is developed based on the zero dynamics concept. With the proposed method, a robust stable walking for a humanoid robot is achieved by adjusting the step timing and landing position of the swing foot automatically, following its intrinsic dynamic characteristics. This exempts the robot from the time-consuming high-level control approaches, especially when a full dynamic model is applied. How different walking patterns/features (i.e., the swing foot motion, the vertical centre of mass motion, the switching manifold configuration, etc.) affect the stability of the walking gait is analysed. Simulations are conducted on robots Romeo and TALOS to support the results.
{"title":"A self-stabilised walking gait for humanoid robots based on the essential model with internal states","authors":"Qiuyue Luo, Christine Chevallereau, Yongsheng Ou, Jianxin Pang, Victor De-León-Gómez, Yannick Aoustin","doi":"10.1049/csy2.12071","DOIUrl":"10.1049/csy2.12071","url":null,"abstract":"<p>Walking stability is one of the key issues for humanoid robots. A self-stabilised walking gait for a full dynamic model of humanoid robots is proposed. For simplified models, that is, the linear inverted pendulum model and variable-length inverted pendulum model, self-stabilisation of walking gait can be obtained if virtual constraints are properly defined. This result is extended to the full dynamic model of humanoid robots by using an essential dynamic model, which is developed based on the zero dynamics concept. With the proposed method, a robust stable walking for a humanoid robot is achieved by adjusting the step timing and landing position of the swing foot automatically, following its intrinsic dynamic characteristics. This exempts the robot from the time-consuming high-level control approaches, especially when a full dynamic model is applied. How different walking patterns/features (i.e., the swing foot motion, the vertical centre of mass motion, the switching manifold configuration, etc.) affect the stability of the walking gait is analysed. Simulations are conducted on robots Romeo and TALOS to support the results.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 4","pages":"283-297"},"PeriodicalIF":0.0,"publicationDate":"2022-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12071","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45234259","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Keeping balance in movement is an important premise for biped robots to complete various tasks. Now, the balance control of biped robots mainly depends on the cooperation of various joints of the robot's body. When robots move faster, the adjustment allowance of joints is reduced, and the robot's anti-disturbance ability will inevitably decline. To solve this problem, the control moment gyroscope (CMG) is creatively used as an auxiliary stabilisation device for fully actuated biped robots and the CMG assistance strategy, which can be integrated into the biped's balance control framework, is proposed. This strategy includes model predictive control module, distribution module, and CMG precession controller. Under the command of it, CMGs can effectively assist the robot in resisting impact and returning to initial positions in time. The results of anti-impact simulation on the walking and running biped robot prove that, with the help of CMGs, the robot's ability to resist disturbance and remain stable is significantly improved.
The cover image is based on the Original Article Disturbance rejection for biped robots during walking and running using control moment gyroscopes by Haochen Xu et al., https://doi.org/10.1049/csy2.12070.
在运动中保持平衡是双足机器人完成各种任务的重要前提。目前,双足机器人的平衡控制主要依赖于机器人身体各关节的配合。当机器人运动速度变快时,关节的调节余量减小,机器人的抗干扰能力必然下降。为了解决这一问题,创造性地将控制力矩陀螺仪(CMG)作为全驱动双足机器人的辅助稳定装置,并提出了将控制力矩陀螺仪辅助策略集成到双足机器人的平衡控制框架中。该策略包括模型预测控制模块、分布模块和CMG进动控制器。在它的指挥下,cmg可以有效地辅助机器人抵抗冲击并及时返回到初始位置。对行走和奔跑两足机器人的抗冲击仿真结果证明,在CMGs的帮助下,机器人的抗干扰和保持稳定的能力得到了显著提高。封面图片来源于Haochen Xu et al., https://doi.org/10.1049/csy2.12070的文章《利用控制力矩陀螺仪抑制双足机器人行走和奔跑过程中的扰动》。
{"title":"Disturbance rejection for biped robots during walking and running using control moment gyroscopes","authors":"Haochen Xu, Zhangguo Yu, Xuechao Chen, Chencheng Dong, Huanzhong Chen, Qiang Huang","doi":"10.1049/csy2.12070","DOIUrl":"10.1049/csy2.12070","url":null,"abstract":"<p>Keeping balance in movement is an important premise for biped robots to complete various tasks. Now, the balance control of biped robots mainly depends on the cooperation of various joints of the robot's body. When robots move faster, the adjustment allowance of joints is reduced, and the robot's anti-disturbance ability will inevitably decline. To solve this problem, the control moment gyroscope (CMG) is creatively used as an auxiliary stabilisation device for fully actuated biped robots and the CMG assistance strategy, which can be integrated into the biped's balance control framework, is proposed. This strategy includes model predictive control module, distribution module, and CMG precession controller. Under the command of it, CMGs can effectively assist the robot in resisting impact and returning to initial positions in time. The results of anti-impact simulation on the walking and running biped robot prove that, with the help of CMGs, the robot's ability to resist disturbance and remain stable is significantly improved.</p><p>The cover image is based on the Original Article <i>Disturbance rejection for biped robots during walking and running using control moment gyroscopes</i> by Haochen Xu et al., https://doi.org/10.1049/csy2.12070.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 4","pages":"268-282"},"PeriodicalIF":0.0,"publicationDate":"2022-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12070","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46888522","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The position control problem of differential-driven automated guided vehicles (AGVs) based on the prescribed-time control method is studied. First, an innovative orientation error function is proposed by an auxiliary arcsine function about the orientation angle. Thus, the problem of position control of AGV is transformed into the stabilisation control of the kinematic system. Second, by introducing a reserved time parameter and a smooth switching function, a novel time-varying scaling function is proposed. This novel scaling function avoids the risk of infinite gain in the conventional prescribed-time control method while ensuring the smoothness of control laws. Then, an improved velocity constraint function is proposed using the Gaussian function. Compared with the existing constraint function, the proposed method not only has more smoothness but also solves the balance point errors caused by the large AGV orientation errors. The presented method ensures that the AGV reaches the target position in a prescribed time. Hence, the upper bound of the AGV system state can be determined by adjusting parameters. Matlab simulation results show that the proposed controller can effectively make the AGV system state satisfy the prescribed bound.
{"title":"Prescribed-time stabilisation control of differential driven automated guided vehicle","authors":"Qiyuan Chen, Pengfei Zhang","doi":"10.1049/csy2.12067","DOIUrl":"https://doi.org/10.1049/csy2.12067","url":null,"abstract":"<p>The position control problem of differential-driven automated guided vehicles (AGVs) based on the prescribed-time control method is studied. First, an innovative orientation error function is proposed by an auxiliary arcsine function about the orientation angle. Thus, the problem of position control of AGV is transformed into the stabilisation control of the kinematic system. Second, by introducing a reserved time parameter and a smooth switching function, a novel time-varying scaling function is proposed. This novel scaling function avoids the risk of infinite gain in the conventional prescribed-time control method while ensuring the smoothness of control laws. Then, an improved velocity constraint function is proposed using the Gaussian function. Compared with the existing constraint function, the proposed method not only has more smoothness but also solves the balance point errors caused by the large AGV orientation errors. The presented method ensures that the AGV reaches the target position in a prescribed time. Hence, the upper bound of the AGV system state can be determined by adjusting parameters. Matlab simulation results show that the proposed controller can effectively make the AGV system state satisfy the prescribed bound.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12067","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"50145046","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Linqi Ye, Jiatai Guo, Jiayi Li, Houde Liu, Xueqian Wang, Bin Liang
This study presents the overall architecture of HeterBot, a heterogeneous mobile manipulation robot developed in our lab, which is designed for versatile operation in hazardous environments. The most significant feature of HeterBot is the heterogeneous design created by adopting the idea of ‘big arm + small arm’ and ‘big car + mini car’. This combination has the advantage of functional complementation, which achieves performance promotion in both locomotion and manipulation capabilities, making HeterBot distinguished from other mobile manipulation robots. Besides, multiple novel technologies are integrated into HeterBot to expand its versatility and ease of use, including Virtual Robot Experimentation Platform-based teleoperation, reconfigurable end effectors, laser-aided grasping, and manipulation with customised tools. Experimental results validate the effectiveness of HeterBot in various locomotion and manipulation tasks. HeterBot displays huge potential in future applications, such as searching and rescue.
{"title":"HeterBot: A heterogeneous mobile manipulation robot for versatile operation","authors":"Linqi Ye, Jiatai Guo, Jiayi Li, Houde Liu, Xueqian Wang, Bin Liang","doi":"10.1049/csy2.12068","DOIUrl":"10.1049/csy2.12068","url":null,"abstract":"<p>This study presents the overall architecture of HeterBot, a heterogeneous mobile manipulation robot developed in our lab, which is designed for versatile operation in hazardous environments. The most significant feature of HeterBot is the heterogeneous design created by adopting the idea of ‘big arm + small arm’ and ‘big car + mini car’. This combination has the advantage of functional complementation, which achieves performance promotion in both locomotion and manipulation capabilities, making HeterBot distinguished from other mobile manipulation robots. Besides, multiple novel technologies are integrated into HeterBot to expand its versatility and ease of use, including Virtual Robot Experimentation Platform-based teleoperation, reconfigurable end effectors, laser-aided grasping, and manipulation with customised tools. Experimental results validate the effectiveness of HeterBot in various locomotion and manipulation tasks. HeterBot displays huge potential in future applications, such as searching and rescue.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-11-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12068","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43011122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recurrent Neural Network, Long Short-Term Memory, and Transformer have made great progress in predicting the trajectories of moving objects. Although the trajectory element with the surrounding scene features has been merged to improve performance, there still exist some problems to be solved. One is that the time series processing models will increase the inference time with the increase of the number of prediction sequences. Another problem is that the features cannot be extracted from the scene's image and point cloud in some situations. Therefore, an Obstacle-Transformer is proposed to predict trajectory in a constant inference time. An ‘obstacle’ is designed by the surrounding trajectory rather than images or point clouds, making Obstacle-Transformer more applicable in a wider range of scenarios. Experiments are conducted on ETH and UCY datasets to verify the performance of our model.
{"title":"Obstacle-transformer: A trajectory prediction network based on surrounding trajectories","authors":"Wendong Zhang, Qingjie Chai, Quanqi Zhang, Chengwei Wu","doi":"10.1049/csy2.12066","DOIUrl":"https://doi.org/10.1049/csy2.12066","url":null,"abstract":"<p>Recurrent Neural Network, Long Short-Term Memory, and Transformer have made great progress in predicting the trajectories of moving objects. Although the trajectory element with the surrounding scene features has been merged to improve performance, there still exist some problems to be solved. One is that the time series processing models will increase the inference time with the increase of the number of prediction sequences. Another problem is that the features cannot be extracted from the scene's image and point cloud in some situations. Therefore, an Obstacle-Transformer is proposed to predict trajectory in a constant inference time. An ‘obstacle’ is designed by the surrounding trajectory rather than images or point clouds, making Obstacle-Transformer more applicable in a wider range of scenarios. Experiments are conducted on ETH and UCY datasets to verify the performance of our model.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12066","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"50140299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Chaoyue Xu, Feifei Qin, Kun Zhou, Binrui Wang, Yinglian Jin
The bionic joints composed of pneumatic muscles (PMs) can simulate the motion of biological joints. However, the PMs themselves have non-linear characteristics such as hysteresis and creep, which make it difficult to achieve high-precision trajectory tracking control of PM-driven robots. In order to effectively suppress the adverse effects of non-linearity on control performance and improve the dynamic performance of PM-driven legged robot, this study designs a double closed-loop control structure based on neural network. First, according to the motion model of the bionic joint, a mapping model between PM contraction force and joint torque is proposed. Second, a control strategy is designed for the inner loop of PM contraction force and the outer loop of bionic joint angle. In the inner control loop, a feedforward neuron Proportional-Integral-Derivative controller is designed based on the PM three-element model. In the control outer loop, a sliding mode robust controller with local model approximation is designed by using the radial basis function neural network approximation capability. Finally, it is verified by simulation and physical experiments that the designed control strategy is suitable for humanoid motion control of antagonistic PM joints, and it can satisfy the requirements of reliability, flexibility, and bionics during human–robot collaboration.
{"title":"A new control for the pneumatic muscle bionic legged robot based on neural network","authors":"Chaoyue Xu, Feifei Qin, Kun Zhou, Binrui Wang, Yinglian Jin","doi":"10.1049/csy2.12065","DOIUrl":"10.1049/csy2.12065","url":null,"abstract":"<p>The bionic joints composed of pneumatic muscles (PMs) can simulate the motion of biological joints. However, the PMs themselves have non-linear characteristics such as hysteresis and creep, which make it difficult to achieve high-precision trajectory tracking control of PM-driven robots. In order to effectively suppress the adverse effects of non-linearity on control performance and improve the dynamic performance of PM-driven legged robot, this study designs a double closed-loop control structure based on neural network. First, according to the motion model of the bionic joint, a mapping model between PM contraction force and joint torque is proposed. Second, a control strategy is designed for the inner loop of PM contraction force and the outer loop of bionic joint angle. In the inner control loop, a feedforward neuron Proportional-Integral-Derivative controller is designed based on the PM three-element model. In the control outer loop, a sliding mode robust controller with local model approximation is designed by using the radial basis function neural network approximation capability. Finally, it is verified by simulation and physical experiments that the designed control strategy is suitable for humanoid motion control of antagonistic PM joints, and it can satisfy the requirements of reliability, flexibility, and bionics during human–robot collaboration.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 4","pages":"339-355"},"PeriodicalIF":0.0,"publicationDate":"2022-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12065","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49382381","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The realisation of a model-based controller for a robot with a higher degree of freedom requires a substantial amount of computational power. A high-speed CPU is required to maintain a higher sampling rate. Multicore processors cannot boost the performance or reduce the execution time as the programs are sequentially structured. The neural network is a great tool to convert a sequentially structured program to an equivalent parallel architecture program. In this study, a radial basis function (RBF) neural network is developed for controlling 7 degrees of freedom of the human lower extremity exoskeleton robot. A realistic friction model is used for modelling joint friction. High trajectory tracking accuracies have been obtained. Evidence of computational efficiency has been observed. The stability analysis of the developed controller is presented. Analysis of variance is used to assess the controller's resilience to parameter variation. To show the effectiveness of the developed controller, a comparative study was performe between the developed RBF network-based controller and Sliding Mode Controller, Computed Torque Controller, Adaptive controller, Linear Quadratic Regulator and Model Reference Computed Torque Controller.
{"title":"Radial basis function-based exoskeleton robot controller development","authors":"SK Hasan","doi":"10.1049/csy2.12057","DOIUrl":"10.1049/csy2.12057","url":null,"abstract":"<p>The realisation of a model-based controller for a robot with a higher degree of freedom requires a substantial amount of computational power. A high-speed CPU is required to maintain a higher sampling rate. Multicore processors cannot boost the performance or reduce the execution time as the programs are sequentially structured. The neural network is a great tool to convert a sequentially structured program to an equivalent parallel architecture program. In this study, a radial basis function (RBF) neural network is developed for controlling 7 degrees of freedom of the human lower extremity exoskeleton robot. A realistic friction model is used for modelling joint friction. High trajectory tracking accuracies have been obtained. Evidence of computational efficiency has been observed. The stability analysis of the developed controller is presented. Analysis of variance is used to assess the controller's resilience to parameter variation. To show the effectiveness of the developed controller, a comparative study was performe between the developed RBF network-based controller and Sliding Mode Controller, Computed Torque Controller, Adaptive controller, Linear Quadratic Regulator and Model Reference Computed Torque Controller.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 3","pages":"228-250"},"PeriodicalIF":0.0,"publicationDate":"2022-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12057","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45108142","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bounding is one of the important gaits in quadrupedal locomotion for negotiating obstacles. The authors proposed an effective approach that can learn robust bounding gaits more efficiently despite its large variation in dynamic body movements. The authors first pretrained the neural network (NN) based on data from a robot operated by conventional model-based controllers, and then further optimised the pretrained NN via deep reinforcement learning (DRL). In particular, the authors designed a reward function considering contact points and phases to enforce the gait symmetry and periodicity, which improved the bounding performance. The NN-based feedback controller was learned in the simulation and directly deployed on the real quadruped robot Jueying Mini successfully. A variety of environments are presented both indoors and outdoors with the authors’ approach. The authors’ approach shows efficient computing and good locomotion results by the Jueying Mini quadrupedal robot bounding over uneven terrain.
The cover image is based on the Research Article Efficient learning of robust quadruped bounding using pretrained neural networks by Zhicheng Wang et al., https://doi.org/10.1049/csy2.12062.
跳跃是四足运动中跨越障碍物的重要步态之一。作者提出了一种有效的方法,可以更有效地学习鲁棒边界步态,尽管它在动态身体运动中变化很大。作者首先根据传统的基于模型的控制器操作的机器人的数据对神经网络(NN)进行预训练,然后通过深度强化学习(DRL)进一步优化预训练的神经网络。特别地,作者设计了一个考虑接触点和相位的奖励函数来增强步态的对称性和周期性,提高了边界性能。在仿真中学习了基于神经网络的反馈控制器,并成功地将其直接部署在真实的四足机器人觉营Mini上。通过作者的方法,呈现了室内和室外的各种环境。该方法证明了聚影迷你四足机器人在不平坦地形上跳跃的计算效率和良好的运动效果。封面图像基于Wang Zhicheng et al., https://doi.org/10.1049/csy2.12062的研究文章《高效学习鲁棒四足动物边界使用预训练神经网络》。
{"title":"Efficient learning of robust quadruped bounding using pretrained neural networks","authors":"Zhicheng Wang, Anqiao Li, Yixiao Zheng, Anhuan Xie, Zhibin Li, Jun Wu, Qiuguo Zhu","doi":"10.1049/csy2.12062","DOIUrl":"10.1049/csy2.12062","url":null,"abstract":"<p>Bounding is one of the important gaits in quadrupedal locomotion for negotiating obstacles. The authors proposed an effective approach that can learn robust bounding gaits more efficiently despite its large variation in dynamic body movements. The authors first pretrained the neural network (NN) based on data from a robot operated by conventional model-based controllers, and then further optimised the pretrained NN via deep reinforcement learning (DRL). In particular, the authors designed a reward function considering contact points and phases to enforce the gait symmetry and periodicity, which improved the bounding performance. The NN-based feedback controller was learned in the simulation and directly deployed on the real quadruped robot Jueying Mini successfully. A variety of environments are presented both indoors and outdoors with the authors’ approach. The authors’ approach shows efficient computing and good locomotion results by the Jueying Mini quadrupedal robot bounding over uneven terrain.</p><p>The cover image is based on the Research Article <i>Efficient learning of robust quadruped bounding using pretrained neural networks</i> by Zhicheng Wang et al., https://doi.org/10.1049/csy2.12062.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 4","pages":"331-338"},"PeriodicalIF":0.0,"publicationDate":"2022-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12062","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46689991","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The semantic representation of the trajectory is conducive to enrich the content of trajectory data mining. A trajectory summarisation generation method based on the mobile robot behaviour analysis was proposed to realize the abstract expression and semantic representation of the spatio-temporal motion features of the robot and its environmental interaction state. First, the behavioural semantic modelling and representation of the mobile robot are completed by modelling the sub-trajectory and calculating the topological behaviour (TOP). Second, Chinese word segmentation and semantic slot filling methods are used to combine with hierarchical clustering to perform basic word extraction and classification for describing trajectory sentences. Then, the description language frame is extracted based on the TOP, and the final trajectory summarisation is generated. The result shows that the proposed method can semantically represent robot behaviours with different motion features and topological features, extract two verb-frameworks for describing the sentences according to their topological features, and dynamically adjust the syntactic structure for the different topological behaviours between the target and the environment. The proposed method can generate semantic information of relatively high quality for spatio-temporal data and help to understand the higher-order semantics of moving robot behaviour.
{"title":"A trajectory summarisation generation method based on the mobile robot behaviour analysis","authors":"Weifeng Liu, Liwen Ma, Shaoyong Qu, Zhangming Peng","doi":"10.1049/csy2.12063","DOIUrl":"10.1049/csy2.12063","url":null,"abstract":"<p>The semantic representation of the trajectory is conducive to enrich the content of trajectory data mining. A trajectory summarisation generation method based on the mobile robot behaviour analysis was proposed to realize the abstract expression and semantic representation of the spatio-temporal motion features of the robot and its environmental interaction state. First, the behavioural semantic modelling and representation of the mobile robot are completed by modelling the sub-trajectory and calculating the topological behaviour (TOP). Second, Chinese word segmentation and semantic slot filling methods are used to combine with hierarchical clustering to perform basic word extraction and classification for describing trajectory sentences. Then, the description language frame is extracted based on the TOP, and the final trajectory summarisation is generated. The result shows that the proposed method can semantically represent robot behaviours with different motion features and topological features, extract two verb-frameworks for describing the sentences according to their topological features, and dynamically adjust the syntactic structure for the different topological behaviours between the target and the environment. The proposed method can generate semantic information of relatively high quality for spatio-temporal data and help to understand the higher-order semantics of moving robot behaviour.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12063","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42549546","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Asynchronous advantage actor-critic (A3C) algorithm is a commonly used policy optimization algorithm in reinforcement learning, in which asynchronous is parallel interactive sampling and training, and advantage is a sampling multi-step reward estimation method for computing weights. In order to address the problem of low efficiency and insufficient convergence caused by the traditional heuristic exploration of A3C algorithm in reinforcement learning, an improved A3C algorithm is proposed in this paper. In this algorithm, a noise network function, which updates the noise tensor in an explicit way is constructed to train the agent. Generalised advantage estimation (GAE) is also adopted to describe the dominance function. Finally, a new mean gradient parallelisation method is designed to update the parameters in both the primary and secondary networks by summing and averaging the gradients passed from all the sub-processes to the main process. Simulation experiments were conducted in a gym environment using the PyTorch Agent Net (PTAN) advanced reinforcement learning library, and the results show that the method enables the agent to complete the learning training faster and its convergence during the training process is better. The improved A3C algorithm has a better performance than the original algorithm, which can provide new ideas for subsequent research on reinforcement learning algorithms.
异步优势actor-critic (A3C)算法是强化学习中常用的策略优化算法,其中异步是并行交互采样和训练,优势是一种计算权重的采样多步奖励估计方法。针对传统的启发式A3C算法在强化学习中效率低、收敛性不足的问题,本文提出了一种改进的A3C算法。该算法通过构造一个噪声网络函数,以显式方式更新噪声张量来训练智能体。采用广义优势估计(GAE)来描述优势函数。最后,设计了一种新的平均梯度并行化方法,通过对所有子过程传递给主过程的梯度求和和平均,来更新主、次网络中的参数。利用PyTorch Agent Net (PTAN)高级强化学习库在体育馆环境下进行了仿真实验,结果表明该方法能够使智能体更快地完成学习训练,并且在训练过程中的收敛性更好。改进后的A3C算法性能优于原算法,可以为后续强化学习算法的研究提供新的思路。
{"title":"A new noise network and gradient parallelisation-based asynchronous advantage actor-critic algorithm","authors":"Zhengshun Fei, Yanping Wang, Jinglong Wang, Kangling Liu, Bingqiang Huang, Ping Tan","doi":"10.1049/csy2.12059","DOIUrl":"10.1049/csy2.12059","url":null,"abstract":"<p>Asynchronous advantage actor-critic (A3C) algorithm is a commonly used policy optimization algorithm in reinforcement learning, in which asynchronous is parallel interactive sampling and training, and advantage is a sampling multi-step reward estimation method for computing weights. In order to address the problem of low efficiency and insufficient convergence caused by the traditional heuristic exploration of A3C algorithm in reinforcement learning, an improved A3C algorithm is proposed in this paper. In this algorithm, a noise network function, which updates the noise tensor in an explicit way is constructed to train the agent. Generalised advantage estimation (GAE) is also adopted to describe the dominance function. Finally, a new mean gradient parallelisation method is designed to update the parameters in both the primary and secondary networks by summing and averaging the gradients passed from all the sub-processes to the main process. Simulation experiments were conducted in a gym environment using the PyTorch Agent Net (PTAN) advanced reinforcement learning library, and the results show that the method enables the agent to complete the learning training faster and its convergence during the training process is better. The improved A3C algorithm has a better performance than the original algorithm, which can provide new ideas for subsequent research on reinforcement learning algorithms.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 3","pages":"175-188"},"PeriodicalIF":0.0,"publicationDate":"2022-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12059","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47988721","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}