首页 > 最新文献

IET Cybersystems and Robotics最新文献

英文 中文
HeterBot: A heterogeneous mobile manipulation robot for versatile operation HeterBot:一种用于多功能操作的异构移动操作机器人
Q3 AUTOMATION & CONTROL SYSTEMS Pub Date : 2022-11-03 DOI: 10.1049/csy2.12068
Linqi Ye, Jiatai Guo, Jiayi Li, Houde Liu, Xueqian Wang, Bin Liang

This study presents the overall architecture of HeterBot, a heterogeneous mobile manipulation robot developed in our lab, which is designed for versatile operation in hazardous environments. The most significant feature of HeterBot is the heterogeneous design created by adopting the idea of ‘big arm + small arm’ and ‘big car + mini car’. This combination has the advantage of functional complementation, which achieves performance promotion in both locomotion and manipulation capabilities, making HeterBot distinguished from other mobile manipulation robots. Besides, multiple novel technologies are integrated into HeterBot to expand its versatility and ease of use, including Virtual Robot Experimentation Platform-based teleoperation, reconfigurable end effectors, laser-aided grasping, and manipulation with customised tools. Experimental results validate the effectiveness of HeterBot in various locomotion and manipulation tasks. HeterBot displays huge potential in future applications, such as searching and rescue.

本研究介绍了我们实验室开发的异构移动操作机器人HeterBot的整体架构,该机器人设计用于在危险环境中进行多种操作。HeterBot最大的特点是采用“大臂+小臂”、“大车+迷你车”的思路,实现了异构设计。这种组合具有功能互补的优势,在运动和操作能力上都实现了性能提升,使HeterBot区别于其他移动操作机器人。此外,多种新技术集成到HeterBot中,以扩展其多功能性和易用性,包括基于虚拟机器人实验平台的远程操作,可重构末端执行器,激光辅助抓取和定制工具操作。实验结果验证了HeterBot在各种运动和操作任务中的有效性。HeterBot在未来的应用中显示出巨大的潜力,比如搜索和救援。
{"title":"HeterBot: A heterogeneous mobile manipulation robot for versatile operation","authors":"Linqi Ye,&nbsp;Jiatai Guo,&nbsp;Jiayi Li,&nbsp;Houde Liu,&nbsp;Xueqian Wang,&nbsp;Bin Liang","doi":"10.1049/csy2.12068","DOIUrl":"10.1049/csy2.12068","url":null,"abstract":"<p>This study presents the overall architecture of HeterBot, a heterogeneous mobile manipulation robot developed in our lab, which is designed for versatile operation in hazardous environments. The most significant feature of HeterBot is the heterogeneous design created by adopting the idea of ‘big arm + small arm’ and ‘big car + mini car’. This combination has the advantage of functional complementation, which achieves performance promotion in both locomotion and manipulation capabilities, making HeterBot distinguished from other mobile manipulation robots. Besides, multiple novel technologies are integrated into HeterBot to expand its versatility and ease of use, including Virtual Robot Experimentation Platform-based teleoperation, reconfigurable end effectors, laser-aided grasping, and manipulation with customised tools. Experimental results validate the effectiveness of HeterBot in various locomotion and manipulation tasks. HeterBot displays huge potential in future applications, such as searching and rescue.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-11-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12068","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43011122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Obstacle-transformer: A trajectory prediction network based on surrounding trajectories 障碍物变换器:一种基于周围轨迹的轨迹预测网络
Q3 AUTOMATION & CONTROL SYSTEMS Pub Date : 2022-10-21 DOI: 10.1049/csy2.12066
Wendong Zhang, Qingjie Chai, Quanqi Zhang, Chengwei Wu

Recurrent Neural Network, Long Short-Term Memory, and Transformer have made great progress in predicting the trajectories of moving objects. Although the trajectory element with the surrounding scene features has been merged to improve performance, there still exist some problems to be solved. One is that the time series processing models will increase the inference time with the increase of the number of prediction sequences. Another problem is that the features cannot be extracted from the scene's image and point cloud in some situations. Therefore, an Obstacle-Transformer is proposed to predict trajectory in a constant inference time. An ‘obstacle’ is designed by the surrounding trajectory rather than images or point clouds, making Obstacle-Transformer more applicable in a wider range of scenarios. Experiments are conducted on ETH and UCY datasets to verify the performance of our model.

递归神经网络、长短期记忆和Transformer在预测运动物体的轨迹方面取得了巨大进展。尽管已经将轨迹元素与周围场景特征合并以提高性能,但仍存在一些问题需要解决。一种是时间序列处理模型会随着预测序列数量的增加而增加推理时间。另一个问题是,在某些情况下,无法从场景的图像和点云中提取特征。因此,提出了一种障碍变换器来预测恒定推理时间内的轨迹。“障碍物”是由周围的轨迹而不是图像或点云设计的,这使得“障碍物变换器”更适用于更广泛的场景。在ETH和UCY数据集上进行了实验,以验证我们模型的性能。
{"title":"Obstacle-transformer: A trajectory prediction network based on surrounding trajectories","authors":"Wendong Zhang,&nbsp;Qingjie Chai,&nbsp;Quanqi Zhang,&nbsp;Chengwei Wu","doi":"10.1049/csy2.12066","DOIUrl":"https://doi.org/10.1049/csy2.12066","url":null,"abstract":"<p>Recurrent Neural Network, Long Short-Term Memory, and Transformer have made great progress in predicting the trajectories of moving objects. Although the trajectory element with the surrounding scene features has been merged to improve performance, there still exist some problems to be solved. One is that the time series processing models will increase the inference time with the increase of the number of prediction sequences. Another problem is that the features cannot be extracted from the scene's image and point cloud in some situations. Therefore, an Obstacle-Transformer is proposed to predict trajectory in a constant inference time. An ‘obstacle’ is designed by the surrounding trajectory rather than images or point clouds, making Obstacle-Transformer more applicable in a wider range of scenarios. Experiments are conducted on ETH and UCY datasets to verify the performance of our model.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12066","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"50140299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A new control for the pneumatic muscle bionic legged robot based on neural network 基于神经网络的气动肌肉仿生腿机器人新控制
Q3 AUTOMATION & CONTROL SYSTEMS Pub Date : 2022-10-09 DOI: 10.1049/csy2.12065
Chaoyue Xu, Feifei Qin, Kun Zhou, Binrui Wang, Yinglian Jin

The bionic joints composed of pneumatic muscles (PMs) can simulate the motion of biological joints. However, the PMs themselves have non-linear characteristics such as hysteresis and creep, which make it difficult to achieve high-precision trajectory tracking control of PM-driven robots. In order to effectively suppress the adverse effects of non-linearity on control performance and improve the dynamic performance of PM-driven legged robot, this study designs a double closed-loop control structure based on neural network. First, according to the motion model of the bionic joint, a mapping model between PM contraction force and joint torque is proposed. Second, a control strategy is designed for the inner loop of PM contraction force and the outer loop of bionic joint angle. In the inner control loop, a feedforward neuron Proportional-Integral-Derivative controller is designed based on the PM three-element model. In the control outer loop, a sliding mode robust controller with local model approximation is designed by using the radial basis function neural network approximation capability. Finally, it is verified by simulation and physical experiments that the designed control strategy is suitable for humanoid motion control of antagonistic PM joints, and it can satisfy the requirements of reliability, flexibility, and bionics during human–robot collaboration.

由气动肌肉组成的仿生关节可以模拟生物关节的运动。然而,由于电机本身具有滞后和蠕变等非线性特性,使得电机驱动机器人难以实现高精度的轨迹跟踪控制。为了有效抑制非线性对控制性能的不利影响,提高pm驱动的腿式机器人的动态性能,本研究设计了一种基于神经网络的双闭环控制结构。首先,根据仿生关节的运动模型,建立了PM收缩力与关节力矩的映射模型;其次,设计了PM收缩力内环和仿生关节角度外环的控制策略;在内部控制回路中,基于PM三元模型设计了前馈神经元比例-积分-导数控制器。在控制外环中,利用径向基函数神经网络的逼近能力,设计了具有局部模型逼近的滑模鲁棒控制器。最后,通过仿真和物理实验验证了所设计的控制策略适用于对抗性PM关节的类人运动控制,能够满足人机协作对可靠性、灵活性和仿生性的要求。
{"title":"A new control for the pneumatic muscle bionic legged robot based on neural network","authors":"Chaoyue Xu,&nbsp;Feifei Qin,&nbsp;Kun Zhou,&nbsp;Binrui Wang,&nbsp;Yinglian Jin","doi":"10.1049/csy2.12065","DOIUrl":"10.1049/csy2.12065","url":null,"abstract":"<p>The bionic joints composed of pneumatic muscles (PMs) can simulate the motion of biological joints. However, the PMs themselves have non-linear characteristics such as hysteresis and creep, which make it difficult to achieve high-precision trajectory tracking control of PM-driven robots. In order to effectively suppress the adverse effects of non-linearity on control performance and improve the dynamic performance of PM-driven legged robot, this study designs a double closed-loop control structure based on neural network. First, according to the motion model of the bionic joint, a mapping model between PM contraction force and joint torque is proposed. Second, a control strategy is designed for the inner loop of PM contraction force and the outer loop of bionic joint angle. In the inner control loop, a feedforward neuron Proportional-Integral-Derivative controller is designed based on the PM three-element model. In the control outer loop, a sliding mode robust controller with local model approximation is designed by using the radial basis function neural network approximation capability. Finally, it is verified by simulation and physical experiments that the designed control strategy is suitable for humanoid motion control of antagonistic PM joints, and it can satisfy the requirements of reliability, flexibility, and bionics during human–robot collaboration.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 4","pages":"339-355"},"PeriodicalIF":0.0,"publicationDate":"2022-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12065","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49382381","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Radial basis function-based exoskeleton robot controller development 基于径向基函数的外骨骼机器人控制器开发
Q3 AUTOMATION & CONTROL SYSTEMS Pub Date : 2022-09-27 DOI: 10.1049/csy2.12057
SK Hasan

The realisation of a model-based controller for a robot with a higher degree of freedom requires a substantial amount of computational power. A high-speed CPU is required to maintain a higher sampling rate. Multicore processors cannot boost the performance or reduce the execution time as the programs are sequentially structured. The neural network is a great tool to convert a sequentially structured program to an equivalent parallel architecture program. In this study, a radial basis function (RBF) neural network is developed for controlling 7 degrees of freedom of the human lower extremity exoskeleton robot. A realistic friction model is used for modelling joint friction. High trajectory tracking accuracies have been obtained. Evidence of computational efficiency has been observed. The stability analysis of the developed controller is presented. Analysis of variance is used to assess the controller's resilience to parameter variation. To show the effectiveness of the developed controller, a comparative study was performe between the developed RBF network-based controller and Sliding Mode Controller, Computed Torque Controller, Adaptive controller, Linear Quadratic Regulator and Model Reference Computed Torque Controller.

为具有更高自由度的机器人实现基于模型的控制器需要大量的计算能力。为了保持较高的采样率,需要高速的CPU。多核处理器不能提高性能或减少执行时间,因为程序是顺序结构化的。神经网络是将顺序结构程序转换为等效并行结构程序的有力工具。本研究开发了一种径向基函数(RBF)神经网络,用于控制人体下肢外骨骼机器人的7个自由度。采用一种真实的摩擦模型来模拟关节摩擦。获得了较高的弹道跟踪精度。计算效率的证据已经被观察到。对所研制的控制器进行了稳定性分析。方差分析用于评估控制器对参数变化的适应能力。为了证明所开发的控制器的有效性,将所开发的基于RBF网络的控制器与滑模控制器、计算转矩控制器、自适应控制器、线性二次型调节器和模型参考计算转矩控制器进行了比较研究。
{"title":"Radial basis function-based exoskeleton robot controller development","authors":"SK Hasan","doi":"10.1049/csy2.12057","DOIUrl":"10.1049/csy2.12057","url":null,"abstract":"<p>The realisation of a model-based controller for a robot with a higher degree of freedom requires a substantial amount of computational power. A high-speed CPU is required to maintain a higher sampling rate. Multicore processors cannot boost the performance or reduce the execution time as the programs are sequentially structured. The neural network is a great tool to convert a sequentially structured program to an equivalent parallel architecture program. In this study, a radial basis function (RBF) neural network is developed for controlling 7 degrees of freedom of the human lower extremity exoskeleton robot. A realistic friction model is used for modelling joint friction. High trajectory tracking accuracies have been obtained. Evidence of computational efficiency has been observed. The stability analysis of the developed controller is presented. Analysis of variance is used to assess the controller's resilience to parameter variation. To show the effectiveness of the developed controller, a comparative study was performe between the developed RBF network-based controller and Sliding Mode Controller, Computed Torque Controller, Adaptive controller, Linear Quadratic Regulator and Model Reference Computed Torque Controller.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 3","pages":"228-250"},"PeriodicalIF":0.0,"publicationDate":"2022-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12057","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45108142","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Efficient learning of robust quadruped bounding using pretrained neural networks 基于预训练神经网络的鲁棒四足动物边界有效学习
Q3 AUTOMATION & CONTROL SYSTEMS Pub Date : 2022-09-25 DOI: 10.1049/csy2.12062
Zhicheng Wang, Anqiao Li, Yixiao Zheng, Anhuan Xie, Zhibin Li, Jun Wu, Qiuguo Zhu

Bounding is one of the important gaits in quadrupedal locomotion for negotiating obstacles. The authors proposed an effective approach that can learn robust bounding gaits more efficiently despite its large variation in dynamic body movements. The authors first pretrained the neural network (NN) based on data from a robot operated by conventional model-based controllers, and then further optimised the pretrained NN via deep reinforcement learning (DRL). In particular, the authors designed a reward function considering contact points and phases to enforce the gait symmetry and periodicity, which improved the bounding performance. The NN-based feedback controller was learned in the simulation and directly deployed on the real quadruped robot Jueying Mini successfully. A variety of environments are presented both indoors and outdoors with the authors’ approach. The authors’ approach shows efficient computing and good locomotion results by the Jueying Mini quadrupedal robot bounding over uneven terrain.

The cover image is based on the Research Article Efficient learning of robust quadruped bounding using pretrained neural networks by Zhicheng Wang et al., https://doi.org/10.1049/csy2.12062.

跳跃是四足运动中跨越障碍物的重要步态之一。作者提出了一种有效的方法,可以更有效地学习鲁棒边界步态,尽管它在动态身体运动中变化很大。作者首先根据传统的基于模型的控制器操作的机器人的数据对神经网络(NN)进行预训练,然后通过深度强化学习(DRL)进一步优化预训练的神经网络。特别地,作者设计了一个考虑接触点和相位的奖励函数来增强步态的对称性和周期性,提高了边界性能。在仿真中学习了基于神经网络的反馈控制器,并成功地将其直接部署在真实的四足机器人觉营Mini上。通过作者的方法,呈现了室内和室外的各种环境。该方法证明了聚影迷你四足机器人在不平坦地形上跳跃的计算效率和良好的运动效果。封面图像基于Wang Zhicheng et al., https://doi.org/10.1049/csy2.12062的研究文章《高效学习鲁棒四足动物边界使用预训练神经网络》。
{"title":"Efficient learning of robust quadruped bounding using pretrained neural networks","authors":"Zhicheng Wang,&nbsp;Anqiao Li,&nbsp;Yixiao Zheng,&nbsp;Anhuan Xie,&nbsp;Zhibin Li,&nbsp;Jun Wu,&nbsp;Qiuguo Zhu","doi":"10.1049/csy2.12062","DOIUrl":"10.1049/csy2.12062","url":null,"abstract":"<p>Bounding is one of the important gaits in quadrupedal locomotion for negotiating obstacles. The authors proposed an effective approach that can learn robust bounding gaits more efficiently despite its large variation in dynamic body movements. The authors first pretrained the neural network (NN) based on data from a robot operated by conventional model-based controllers, and then further optimised the pretrained NN via deep reinforcement learning (DRL). In particular, the authors designed a reward function considering contact points and phases to enforce the gait symmetry and periodicity, which improved the bounding performance. The NN-based feedback controller was learned in the simulation and directly deployed on the real quadruped robot Jueying Mini successfully. A variety of environments are presented both indoors and outdoors with the authors’ approach. The authors’ approach shows efficient computing and good locomotion results by the Jueying Mini quadrupedal robot bounding over uneven terrain.</p><p>The cover image is based on the Research Article <i>Efficient learning of robust quadruped bounding using pretrained neural networks</i> by Zhicheng Wang et al., https://doi.org/10.1049/csy2.12062.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 4","pages":"331-338"},"PeriodicalIF":0.0,"publicationDate":"2022-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12062","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46689991","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
A trajectory summarisation generation method based on the mobile robot behaviour analysis 基于移动机器人行为分析的轨迹汇总生成方法
Q3 AUTOMATION & CONTROL SYSTEMS Pub Date : 2022-09-23 DOI: 10.1049/csy2.12063
Weifeng Liu, Liwen Ma, Shaoyong Qu, Zhangming Peng

The semantic representation of the trajectory is conducive to enrich the content of trajectory data mining. A trajectory summarisation generation method based on the mobile robot behaviour analysis was proposed to realize the abstract expression and semantic representation of the spatio-temporal motion features of the robot and its environmental interaction state. First, the behavioural semantic modelling and representation of the mobile robot are completed by modelling the sub-trajectory and calculating the topological behaviour (TOP). Second, Chinese word segmentation and semantic slot filling methods are used to combine with hierarchical clustering to perform basic word extraction and classification for describing trajectory sentences. Then, the description language frame is extracted based on the TOP, and the final trajectory summarisation is generated. The result shows that the proposed method can semantically represent robot behaviours with different motion features and topological features, extract two verb-frameworks for describing the sentences according to their topological features, and dynamically adjust the syntactic structure for the different topological behaviours between the target and the environment. The proposed  method can generate semantic information of relatively high quality for spatio-temporal data and help to understand the higher-order semantics of moving robot behaviour.

轨迹的语义表示有利于丰富轨迹数据挖掘的内容。提出了一种基于移动机器人行为分析的轨迹汇总生成方法,实现了机器人时空运动特征及其环境交互状态的抽象表达和语义表示。首先,通过子轨迹建模和拓扑行为计算(TOP)完成移动机器人的行为语义建模和表示。其次,采用汉语分词和语义槽填充方法,结合层次聚类对轨迹句进行基本词提取和分类;然后,基于TOP提取描述语言框架,生成最终的轨迹摘要。结果表明,该方法可以对具有不同运动特征和拓扑特征的机器人行为进行语义表示,根据句子的拓扑特征提取两个动词框架来描述句子,并针对目标和环境之间的不同拓扑行为动态调整句法结构。该方法可以为时空数据生成质量较高的语义信息,有助于理解机器人运动行为的高阶语义。
{"title":"A trajectory summarisation generation method based on the mobile robot behaviour analysis","authors":"Weifeng Liu,&nbsp;Liwen Ma,&nbsp;Shaoyong Qu,&nbsp;Zhangming Peng","doi":"10.1049/csy2.12063","DOIUrl":"10.1049/csy2.12063","url":null,"abstract":"<p>The semantic representation of the trajectory is conducive to enrich the content of trajectory data mining. A trajectory summarisation generation method based on the mobile robot behaviour analysis was proposed to realize the abstract expression and semantic representation of the spatio-temporal motion features of the robot and its environmental interaction state. First, the behavioural semantic modelling and representation of the mobile robot are completed by modelling the sub-trajectory and calculating the topological behaviour (TOP). Second, Chinese word segmentation and semantic slot filling methods are used to combine with hierarchical clustering to perform basic word extraction and classification for describing trajectory sentences. Then, the description language frame is extracted based on the TOP, and the final trajectory summarisation is generated. The result shows that the proposed method can semantically represent robot behaviours with different motion features and topological features, extract two verb-frameworks for describing the sentences according to their topological features, and dynamically adjust the syntactic structure for the different topological behaviours between the target and the environment. The proposed  method can generate semantic information of relatively high quality for spatio-temporal data and help to understand the higher-order semantics of moving robot behaviour.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12063","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42549546","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A new noise network and gradient parallelisation-based asynchronous advantage actor-critic algorithm 一种新的基于噪声网络和梯度并行化的异步优势因子-批评家算法
Q3 AUTOMATION & CONTROL SYSTEMS Pub Date : 2022-09-22 DOI: 10.1049/csy2.12059
Zhengshun Fei, Yanping Wang, Jinglong Wang, Kangling Liu, Bingqiang Huang, Ping Tan

Asynchronous advantage actor-critic (A3C) algorithm is a commonly used policy optimization algorithm in reinforcement learning, in which asynchronous is parallel interactive sampling and training, and advantage is a sampling multi-step reward estimation method for computing weights. In order to address the problem of low efficiency and insufficient convergence caused by the traditional heuristic exploration of A3C algorithm in reinforcement learning, an improved A3C algorithm is proposed in this paper. In this algorithm, a noise network function, which updates the noise tensor in an explicit way is constructed to train the agent. Generalised advantage estimation (GAE) is also adopted to describe the dominance function. Finally, a new mean gradient parallelisation method is designed to update the parameters in both the primary and secondary networks by summing and averaging the gradients passed from all the sub-processes to the main process. Simulation experiments were conducted in a gym environment using the PyTorch Agent Net (PTAN) advanced reinforcement learning library, and the results show that the method enables the agent to complete the learning training faster and its convergence during the training process is better. The improved A3C algorithm has a better performance than the original algorithm, which can provide new ideas for subsequent research on reinforcement learning algorithms.

异步优势actor-critic (A3C)算法是强化学习中常用的策略优化算法,其中异步是并行交互采样和训练,优势是一种计算权重的采样多步奖励估计方法。针对传统的启发式A3C算法在强化学习中效率低、收敛性不足的问题,本文提出了一种改进的A3C算法。该算法通过构造一个噪声网络函数,以显式方式更新噪声张量来训练智能体。采用广义优势估计(GAE)来描述优势函数。最后,设计了一种新的平均梯度并行化方法,通过对所有子过程传递给主过程的梯度求和和平均,来更新主、次网络中的参数。利用PyTorch Agent Net (PTAN)高级强化学习库在体育馆环境下进行了仿真实验,结果表明该方法能够使智能体更快地完成学习训练,并且在训练过程中的收敛性更好。改进后的A3C算法性能优于原算法,可以为后续强化学习算法的研究提供新的思路。
{"title":"A new noise network and gradient parallelisation-based asynchronous advantage actor-critic algorithm","authors":"Zhengshun Fei,&nbsp;Yanping Wang,&nbsp;Jinglong Wang,&nbsp;Kangling Liu,&nbsp;Bingqiang Huang,&nbsp;Ping Tan","doi":"10.1049/csy2.12059","DOIUrl":"10.1049/csy2.12059","url":null,"abstract":"<p>Asynchronous advantage actor-critic (A3C) algorithm is a commonly used policy optimization algorithm in reinforcement learning, in which asynchronous is parallel interactive sampling and training, and advantage is a sampling multi-step reward estimation method for computing weights. In order to address the problem of low efficiency and insufficient convergence caused by the traditional heuristic exploration of A3C algorithm in reinforcement learning, an improved A3C algorithm is proposed in this paper. In this algorithm, a noise network function, which updates the noise tensor in an explicit way is constructed to train the agent. Generalised advantage estimation (GAE) is also adopted to describe the dominance function. Finally, a new mean gradient parallelisation method is designed to update the parameters in both the primary and secondary networks by summing and averaging the gradients passed from all the sub-processes to the main process. Simulation experiments were conducted in a gym environment using the PyTorch Agent Net (PTAN) advanced reinforcement learning library, and the results show that the method enables the agent to complete the learning training faster and its convergence during the training process is better. The improved A3C algorithm has a better performance than the original algorithm, which can provide new ideas for subsequent research on reinforcement learning algorithms.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 3","pages":"175-188"},"PeriodicalIF":0.0,"publicationDate":"2022-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12059","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47988721","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
A heuristic control framework for heavy-duty hexapod robot over complex terrain 复杂地形下重型六足机器人的启发式控制框架
Q3 AUTOMATION & CONTROL SYSTEMS Pub Date : 2022-09-21 DOI: 10.1049/csy2.12064
Jinmian Hou, Hui Chai, Yibin Li, Yaxian Xin, Wei Chen

The large and heavy-duty hexapod robot has strong motion stability and load capacity, which promises to have a wide range of application prospects in rescue and disaster relief. Multi-mode gait and static stability during walking make the hexapod robot adapt to more diverse terrains, while little research has been conducted on the motion control methods of heavy-duty hexapod robots in complex environments. A novel heuristic whole-body motion control framework for the heavy-duty hexapod robot to traverse complex terrain is presented. By splitting the legged locomotion into a single task, the whole-body motion could be planned in a reasonable time. The terrain adaptation strategy is designed to improve the complex terrain passability. Ground reaction forces are then optimised based on single rigid-body dynamics with heuristics. This framework utilised simple but powerful heuristics to approximate complex dynamics and allows for a single set of parameters for all task conditions. Simulation results demonstrate the robustness and adaptability of the proposed framework.

大型重型六足机器人具有较强的运动稳定性和承载能力,在抢险救灾中具有广泛的应用前景。多模式步态和行走时的静态稳定性使六足机器人能够适应更多样化的地形,而重载六足机器人在复杂环境下的运动控制方法研究较少。提出了一种重载六足机器人穿越复杂地形的启发式全身运动控制框架。通过将腿部运动拆分为单个任务,可以在合理的时间内规划全身运动。为提高复杂地形的通过性,设计了地形适应策略。然后基于单刚体动力学启发式优化地面反作用力。该框架利用简单但功能强大的启发式方法来近似复杂的动态,并允许对所有任务条件使用一组参数。仿真结果证明了该框架的鲁棒性和适应性。
{"title":"A heuristic control framework for heavy-duty hexapod robot over complex terrain","authors":"Jinmian Hou,&nbsp;Hui Chai,&nbsp;Yibin Li,&nbsp;Yaxian Xin,&nbsp;Wei Chen","doi":"10.1049/csy2.12064","DOIUrl":"10.1049/csy2.12064","url":null,"abstract":"<p>The large and heavy-duty hexapod robot has strong motion stability and load capacity, which promises to have a wide range of application prospects in rescue and disaster relief. Multi-mode gait and static stability during walking make the hexapod robot adapt to more diverse terrains, while little research has been conducted on the motion control methods of heavy-duty hexapod robots in complex environments. A novel heuristic whole-body motion control framework for the heavy-duty hexapod robot to traverse complex terrain is presented. By splitting the legged locomotion into a single task, the whole-body motion could be planned in a reasonable time. The terrain adaptation strategy is designed to improve the complex terrain passability. Ground reaction forces are then optimised based on single rigid-body dynamics with heuristics. This framework utilised simple but powerful heuristics to approximate complex dynamics and allows for a single set of parameters for all task conditions. Simulation results demonstrate the robustness and adaptability of the proposed framework.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 4","pages":"322-330"},"PeriodicalIF":0.0,"publicationDate":"2022-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12064","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43335590","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Unsupervised learning on particle image velocimetry with embedded cross-correlation and divergence-free constraint 嵌入互相关和无发散约束的粒子图像测速无监督学习
Q3 AUTOMATION & CONTROL SYSTEMS Pub Date : 2022-09-21 DOI: 10.1049/csy2.12056
Yiwei Chong, Jiaming Liang, Tehuan Chen, Chao Xu, Changchun Pan

Particle image velocimetry (PIV) is an essential method in experimental fluid dynamics. In recent years, the development of deep learning-based methods has inspired new approaches to tackle the PIV problem, which considerably improves the accuracy of PIV. However, the supervised learning of PIV is driven by large volumes of data with ground truth information. Therefore, the authors consider unsupervised PIV methods. There has been some work on unsupervised PIV, but they are not nearly as effective as supervised learning PIV. The authors try to improve the effectiveness and accuracy of unsupervised PIV by adding classical PIV methods and physical constraints. In this paper, the authors propose an unsupervised PIV method combined with the cross-correlation method and divergence-free constraint, which obtains better performance than other unsupervised PIV methods. The authors compare some classical PIV methods and some deep learning methods, such as LiteFlowNet, LiteFlowNet-en, and UnLiteFlowNet with the authors’ model on the synthetic dataset. Besides, the authors contrast the results of LiteFlowNet, UnLiteFlowNet and the authors’ model on experimental particle images. As a result, the authors’ model shows comparable performance with classical PIV methods as well as supervised PIV methods and outperforms the previous unsupervised PIV method in most flow cases.

粒子图像测速(PIV)是实验流体力学中的一种重要方法。近年来,基于深度学习的方法的发展激发了解决PIV问题的新方法,大大提高了PIV的准确性。然而,PIV的监督学习是由大量具有地面真实信息的数据驱动的。因此,作者考虑了无监督的PIV方法。已经有一些关于无监督PIV的研究,但它们远不如监督学习PIV有效。作者试图通过加入经典的PIV方法和物理约束来提高无监督PIV的有效性和准确性。本文提出了一种结合互相关方法和无散度约束的无监督PIV方法,该方法取得了比其他无监督PIV方法更好的性能。作者将一些经典的PIV方法和一些深度学习方法(如LiteFlowNet、LiteFlowNet-en和UnLiteFlowNet)与作者在合成数据集上的模型进行了比较。此外,作者还将LiteFlowNet、UnLiteFlowNet和作者模型在实验粒子图像上的结果进行了对比。结果表明,该模型的性能与经典PIV方法和有监督PIV方法相当,并且在大多数流情况下优于之前的无监督PIV方法。
{"title":"Unsupervised learning on particle image velocimetry with embedded cross-correlation and divergence-free constraint","authors":"Yiwei Chong,&nbsp;Jiaming Liang,&nbsp;Tehuan Chen,&nbsp;Chao Xu,&nbsp;Changchun Pan","doi":"10.1049/csy2.12056","DOIUrl":"10.1049/csy2.12056","url":null,"abstract":"<p>Particle image velocimetry (PIV) is an essential method in experimental fluid dynamics. In recent years, the development of deep learning-based methods has inspired new approaches to tackle the PIV problem, which considerably improves the accuracy of PIV. However, the supervised learning of PIV is driven by large volumes of data with ground truth information. Therefore, the authors consider unsupervised PIV methods. There has been some work on unsupervised PIV, but they are not nearly as effective as supervised learning PIV. The authors try to improve the effectiveness and accuracy of unsupervised PIV by adding classical PIV methods and physical constraints. In this paper, the authors propose an unsupervised PIV method combined with the cross-correlation method and divergence-free constraint, which obtains better performance than other unsupervised PIV methods. The authors compare some classical PIV methods and some deep learning methods, such as LiteFlowNet, LiteFlowNet-en, and UnLiteFlowNet with the authors’ model on the synthetic dataset. Besides, the authors contrast the results of LiteFlowNet, UnLiteFlowNet and the authors’ model on experimental particle images. As a result, the authors’ model shows comparable performance with classical PIV methods as well as supervised PIV methods and outperforms the previous unsupervised PIV method in most flow cases.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 3","pages":"200-211"},"PeriodicalIF":0.0,"publicationDate":"2022-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12056","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44997054","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Distributed non-ideal leader estimation and formation control for multiple non-holonomic mobile robots 多个非完整移动机器人的分布式非理想领导者估计与编队控制
Q3 AUTOMATION & CONTROL SYSTEMS Pub Date : 2022-09-19 DOI: 10.1049/csy2.12061
Peifen Lu, Zhigang Ren, Zongze Wu, Zhipeng Li, Shichao Zhou

This paper studies a distributed formation problem for non-holonomic mobile robots. Consideration of the leader dynamics of the robots as non-ideal, that is, subject to disturbances/unmodelled variables, is the distinguishing feature of this work. The issue is resolved by a distributed combined disturbance-and-leader estimator, allowing for the distributed reconstruction of the leader's signals. The estimator needs to detect the leader's information and disturbance. In order to reject such disturbance and achieve the formation asymptotically, the control law incorporates the smooth estimator's estimate of the leader disturbance. Furthermore, the stability of the total distributed formation control algorithm is also examined using the Lyapunov technique. Finally, to show the viability of the proposed theoretical results, simulations and actual experiments are carried out.

研究了一类非完整移动机器人的分布式编队问题。考虑到机器人的前导动力学是非理想的,即受干扰/未建模变量的影响,是这项工作的显著特征。这个问题是通过一个分布式组合干扰和先导估计器来解决的,允许先导信号的分布式重建。估计器需要检测出领导者的信息和干扰。为了抑制这种扰动并使其渐近形成,控制律中加入了光滑估计器对前导扰动的估计。此外,还利用李亚普诺夫技术检验了全分布式编队控制算法的稳定性。最后,为了证明所提理论结果的可行性,进行了仿真和实际实验。
{"title":"Distributed non-ideal leader estimation and formation control for multiple non-holonomic mobile robots","authors":"Peifen Lu,&nbsp;Zhigang Ren,&nbsp;Zongze Wu,&nbsp;Zhipeng Li,&nbsp;Shichao Zhou","doi":"10.1049/csy2.12061","DOIUrl":"10.1049/csy2.12061","url":null,"abstract":"<p>This paper studies a distributed formation problem for non-holonomic mobile robots. Consideration of the leader dynamics of the robots as non-ideal, that is, subject to disturbances/unmodelled variables, is the distinguishing feature of this work. The issue is resolved by a distributed combined disturbance-and-leader estimator, allowing for the distributed reconstruction of the leader's signals. The estimator needs to detect the leader's information and disturbance. In order to reject such disturbance and achieve the formation asymptotically, the control law incorporates the smooth estimator's estimate of the leader disturbance. Furthermore, the stability of the total distributed formation control algorithm is also examined using the Lyapunov technique. Finally, to show the viability of the proposed theoretical results, simulations and actual experiments are carried out.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"4 3","pages":"163-174"},"PeriodicalIF":0.0,"publicationDate":"2022-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12061","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47102191","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
IET Cybersystems and Robotics
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1