Pub Date : 2022-01-01DOI: 10.1016/j.cogr.2022.01.002
Yichen Zhong , Yanfeng Pu , Ting Wang
Aiming at achieving high flexibility and safety, telerehabilitation systems and telesurgery systems often use flexible manipulators in the telerehabilitation systems. However, due to the structure of the flexible manipulator, it has strong model uncertainties and nonlinearity in its dynamic model which causes the difficulty of the accurate control. In order to accomplish accurate trajectory tracking of telerehabilitations systems with flexible manipulators, a bilateral controller is introduced on the basis of the sliding mode control strategy and a non-linear disturbance observer. The non-linear disturbance observer is applied to estimate the model uncertainties and external disturbance of both the master and the slave flexible manipulators in the telerehabilitation system. The asymptotic stability is analyzed by the Lyapunov function. Numerical simulations are performed and results show efficiency and effectiveness of our method.
{"title":"A sliding mode and non-linear disturbance observer based bilateral control for telerehabilitation systems with flexible manipulators","authors":"Yichen Zhong , Yanfeng Pu , Ting Wang","doi":"10.1016/j.cogr.2022.01.002","DOIUrl":"10.1016/j.cogr.2022.01.002","url":null,"abstract":"<div><p>Aiming at achieving high flexibility and safety, telerehabilitation systems and telesurgery systems often use flexible manipulators in the telerehabilitation systems. However, due to the structure of the flexible manipulator, it has strong model uncertainties and nonlinearity in its dynamic model which causes the difficulty of the accurate control. In order to accomplish accurate trajectory tracking of telerehabilitations systems with flexible manipulators, a bilateral controller is introduced on the basis of the sliding mode control strategy and a non-linear disturbance observer. The non-linear disturbance observer is applied to estimate the model uncertainties and external disturbance of both the master and the slave flexible manipulators in the telerehabilitation system. The asymptotic stability is analyzed by the Lyapunov function. Numerical simulations are performed and results show efficiency and effectiveness of our method.</p></div>","PeriodicalId":100288,"journal":{"name":"Cognitive Robotics","volume":"2 ","pages":"Pages 39-49"},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2667241322000027/pdfft?md5=b7d9d250b28663087ed65c328e03f908&pid=1-s2.0-S2667241322000027-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90269622","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-01-01DOI: 10.1016/j.cogr.2022.12.002
Boxiong Yang , Shunmin Wang , Shelei Li , Bo Zhou , Fujun Zhao , Faizan Ali , Hui He
Hyperspectral remote sensing has been an important technical means to obtain more refined information and provide rich, accurate, and reasonable data for the quantitative analysis and delicacy management of a "smart city". To better understand and use the hyperspectral data to help the construction of a digital city, the study of the feature and characteristics of hyperspectral remote sensing images is introduced in this paper. Then how to collect the hyperspectral information of urban ground objects through the unmanned aerial vehicle (UAV) and hyperspectral imager was described, which greatly improves the efficiency of urban data acquisition. Finally, various application cases of UAV-based hyperspectral remote sensing and deep information mining of urban ground objects were analyzed and discussed in detail, such as terrain classification, urban greening analysis, etc. The research result shows that airborne hyperspectral imagery (HIS) has unique advantages over color photography and multispectral remote sensing, with a richer and higher level of spectral details and physical & chemical properties.
{"title":"Research and application of UAV-based hyperspectral remote sensing for smart city construction","authors":"Boxiong Yang , Shunmin Wang , Shelei Li , Bo Zhou , Fujun Zhao , Faizan Ali , Hui He","doi":"10.1016/j.cogr.2022.12.002","DOIUrl":"10.1016/j.cogr.2022.12.002","url":null,"abstract":"<div><p>Hyperspectral remote sensing has been an important technical means to obtain more refined information and provide rich, accurate, and reasonable data for the quantitative analysis and delicacy management of a \"smart city\". To better understand and use the hyperspectral data to help the construction of a digital city, the study of the feature and characteristics of hyperspectral remote sensing images is introduced in this paper. Then how to collect the hyperspectral information of urban ground objects through the unmanned aerial vehicle (UAV) and hyperspectral imager was described, which greatly improves the efficiency of urban data acquisition. Finally, various application cases of UAV-based hyperspectral remote sensing and deep information mining of urban ground objects were analyzed and discussed in detail, such as terrain classification, urban greening analysis, etc. The research result shows that airborne hyperspectral imagery (HIS) has unique advantages over color photography and multispectral remote sensing, with a richer and higher level of spectral details and physical & chemical properties.</p></div>","PeriodicalId":100288,"journal":{"name":"Cognitive Robotics","volume":"2 ","pages":"Pages 255-266"},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2667241322000234/pdfft?md5=c9c10c3ec3160d3471e685148c043c95&pid=1-s2.0-S2667241322000234-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74497174","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-01-01DOI: 10.1016/j.cogr.2022.06.001
Chaofei Yu, Mei Wang
Emotion is an indispensable part of human emotion, which affects human normal physiological activities and daily life decisions. Human emotion recognition is a critical technology in artificial intelligence, human-computer interaction, and other fields. The brain is the information processing and control center of the human body. Electroencephalogram (EEG) physiological signals are generated directly by the central nervous system, closely related to human emotions. Therefore, EEG signals can objectively and now reflect the human emotional state in real-time. In recent years, with the development of the brain-computer interface, the acquisition and analysis technology of human EEG signals has become increasingly mature, so more and more researchers use the research method based on EEG signals to study emotion recognition. EEG processing plays a vital role in emotion recognition. This paper presents a recent research report on emotion recognition. This paper introduces the related analysis methods and research contents from the aspects of emotion induction, EEG preprocessing, feature extraction, and emotion classification and compares the advantages and disadvantages of these methods. This paper summarizes the problems existing in current research methods. This paper discusses the research direction of emotion classification based on EEG information.
{"title":"Survey of emotion recognition methods using EEG information","authors":"Chaofei Yu, Mei Wang","doi":"10.1016/j.cogr.2022.06.001","DOIUrl":"10.1016/j.cogr.2022.06.001","url":null,"abstract":"<div><p>Emotion is an indispensable part of human emotion, which affects human normal physiological activities and daily life decisions. Human emotion recognition is a critical technology in artificial intelligence, human-computer interaction, and other fields. The brain is the information processing and control center of the human body. Electroencephalogram (EEG) physiological signals are generated directly by the central nervous system, closely related to human emotions. Therefore, EEG signals can objectively and now reflect the human emotional state in real-time. In recent years, with the development of the brain-computer interface, the acquisition and analysis technology of human EEG signals has become increasingly mature, so more and more researchers use the research method based on EEG signals to study emotion recognition. EEG processing plays a vital role in emotion recognition. This paper presents a recent research report on emotion recognition. This paper introduces the related analysis methods and research contents from the aspects of emotion induction, EEG preprocessing, feature extraction, and emotion classification and compares the advantages and disadvantages of these methods. This paper summarizes the problems existing in current research methods. This paper discusses the research direction of emotion classification based on EEG information.</p></div>","PeriodicalId":100288,"journal":{"name":"Cognitive Robotics","volume":"2 ","pages":"Pages 132-146"},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2667241322000118/pdfft?md5=2ebed0bccb0121e06426dee7bae45d6f&pid=1-s2.0-S2667241322000118-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86018605","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-01-01DOI: 10.1016/j.cogr.2022.08.002
Peixin Qu , Tengfei Li , Guohou Li , Zhen Tian , Xiwang Xie , Wenyi Zhao , Xipeng Pan , Weidong Zhang
Underwater images are serious problems affected by the absorption and scattering of light. At present, the existing sharpening methods can't effectively solve all underwater image degradation problems, thus it is necessary to propose a specific solution to the degradation problem. To solve the above problems, the Multi-Color Convolutional and Attentional Stacking Network (MCCA-Net) for Underwater image classification are proposed in this paper. First, an underwater image is converted to HSV and Lab color spaces and fused to achieve a refined image. Then, the attention mechanism module is used to fine the extracted image features. At last, the vertically stacked convolution module fully utilizes different levels of feature information, which realizes the fusion of convolution and attention mechanism, optimizes feature extraction and parameter reduction, and improves the classification performance of the MCCA-Net model. Extensive experiments on underwater degraded image classification show that our MCCA-Net model and method outperform other models and improve the accuracy of underwater degraded image classification. Our image fusion method can achieve 96.39% accuracy on other models, and the MCCA-Net model achieves 97.38% classification accuracy.
{"title":"MCCA-Net: Multi-color convolution and attention stacked network for Underwater image classification","authors":"Peixin Qu , Tengfei Li , Guohou Li , Zhen Tian , Xiwang Xie , Wenyi Zhao , Xipeng Pan , Weidong Zhang","doi":"10.1016/j.cogr.2022.08.002","DOIUrl":"10.1016/j.cogr.2022.08.002","url":null,"abstract":"<div><p>Underwater images are serious problems affected by the absorption and scattering of light. At present, the existing sharpening methods can't effectively solve all underwater image degradation problems, thus it is necessary to propose a specific solution to the degradation problem. To solve the above problems, the Multi-Color Convolutional and Attentional Stacking Network (MCCA-Net) for Underwater image classification are proposed in this paper. First, an underwater image is converted to HSV and Lab color spaces and fused to achieve a refined image. Then, the attention mechanism module is used to fine the extracted image features. At last, the vertically stacked convolution module fully utilizes different levels of feature information, which realizes the fusion of convolution and attention mechanism, optimizes feature extraction and parameter reduction, and improves the classification performance of the MCCA-Net model. Extensive experiments on underwater degraded image classification show that our MCCA-Net model and method outperform other models and improve the accuracy of underwater degraded image classification. Our image fusion method can achieve 96.39% accuracy on other models, and the MCCA-Net model achieves 97.38% classification accuracy.</p></div>","PeriodicalId":100288,"journal":{"name":"Cognitive Robotics","volume":"2 ","pages":"Pages 211-221"},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2667241322000192/pdfft?md5=9bb766a2fd8a481c394e42fdefd438ef&pid=1-s2.0-S2667241322000192-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88427619","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recently, two-dimensional object detection (2D object detection) has been introduced in numerous applications such as building exterior diagnosis, crime prevention and surveillance, and medical fields. However, the distance (depth) information is not enough for indoor robot navigation, robot grasping, autonomous running, and so on, with conventional object detection. Therefore, in order to improve the accuracy of 3D object detection, this paper proposes an improvement of Point RCNN, which is a segmentation-based method using RPNs and has performed well in 3D detection benchmarks on the KITTI dataset commonly used in recognition tasks for automatic driving. The proposed improvement is to improve the network in the first stage of generating 3D box candidates in order to solve the problem of frequent false positives. Specifically, we added a Squeeze and Excitation (SE) Block to the network of pointnet++ that performs feature extraction in the first stage and changed the activation function from ReLU to Mish. Experiments were conducted on the KITTI dataset, which is commonly used in research aimed at automated driving, and an accurate comparison was conducted using AP. The proposed method outperforms the conventional method by several percent on all three difficulty levels.
{"title":"3D object detection using improved PointRCNN","authors":"Kazuki Fukitani, Ishiyama Shin, Huimin Lu, Shuo Yang, Tohru Kamiya, Yoshihisa Nakatoh, Seiichi Serikawa","doi":"10.1016/j.cogr.2022.12.001","DOIUrl":"https://doi.org/10.1016/j.cogr.2022.12.001","url":null,"abstract":"<div><p>Recently, two-dimensional object detection (2D object detection) has been introduced in numerous applications such as building exterior diagnosis, crime prevention and surveillance, and medical fields. However, the distance (depth) information is not enough for indoor robot navigation, robot grasping, autonomous running, and so on, with conventional object detection. Therefore, in order to improve the accuracy of 3D object detection, this paper proposes an improvement of Point RCNN, which is a segmentation-based method using RPNs and has performed well in 3D detection benchmarks on the KITTI dataset commonly used in recognition tasks for automatic driving. The proposed improvement is to improve the network in the first stage of generating 3D box candidates in order to solve the problem of frequent false positives. Specifically, we added a Squeeze and Excitation (SE) Block to the network of pointnet++ that performs feature extraction in the first stage and changed the activation function from ReLU to Mish. Experiments were conducted on the KITTI dataset, which is commonly used in research aimed at automated driving, and an accurate comparison was conducted using AP. The proposed method outperforms the conventional method by several percent on all three difficulty levels.</p></div>","PeriodicalId":100288,"journal":{"name":"Cognitive Robotics","volume":"2 ","pages":"Pages 242-254"},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2667241322000222/pdfft?md5=976fa9833e04a5bb9d3751cbbe165535&pid=1-s2.0-S2667241322000222-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"92004295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-01-01DOI: 10.1016/j.cogr.2022.03.003
Shenglin Mu, Satoru Shibata, Tomonori Yamamoto
In this paper, a user-following mobile robot which tracks and follows the user, offering stand-up assistance function is proposed. The proposed robot plays the role of a chair where the user can sit on, and offers a stand-up assistance function compensating the lack of muscle strength. In the proposed robot, a sensing method for buttocks recognition using a depth sensor is proposed. By measuring the distance from the user’s buttocks, the walking state is recognized and the tracking is performed at a fixed distance. As an approach to realize the tracking function, a human tracking method for mobile robots using PD control is constructed. According experimental study, usefulness of the proposed mobile robot with the function of user-following and stand-up assistance is confirmed. The user recognition method and the tracking method using PD control are confirmed effective. With the proposed robot system, improvement in welfare field can be expected.
{"title":"Development of a user-following mobile robot with a stand-up assistance function","authors":"Shenglin Mu, Satoru Shibata, Tomonori Yamamoto","doi":"10.1016/j.cogr.2022.03.003","DOIUrl":"https://doi.org/10.1016/j.cogr.2022.03.003","url":null,"abstract":"<div><p>In this paper, a user-following mobile robot which tracks and follows the user, offering stand-up assistance function is proposed. The proposed robot plays the role of a chair where the user can sit on, and offers a stand-up assistance function compensating the lack of muscle strength. In the proposed robot, a sensing method for buttocks recognition using a depth sensor is proposed. By measuring the distance from the user’s buttocks, the walking state is recognized and the tracking is performed at a fixed distance. As an approach to realize the tracking function, a human tracking method for mobile robots using PD control is constructed. According experimental study, usefulness of the proposed mobile robot with the function of user-following and stand-up assistance is confirmed. The user recognition method and the tracking method using PD control are confirmed effective. With the proposed robot system, improvement in welfare field can be expected.</p></div>","PeriodicalId":100288,"journal":{"name":"Cognitive Robotics","volume":"2 ","pages":"Pages 83-95"},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2667241322000064/pdfft?md5=eb234cb3880e54b18b3ff70643c72736&pid=1-s2.0-S2667241322000064-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"92091649","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-01-01DOI: 10.1016/j.cogr.2021.12.002
Zhuoya Zhang , Fei Xu , Zengshi Qin , Yue Xie
With the progress of power grid technology and intelligent technology, intelligent inspection robot (IR) came into being and are expected to become the main force of substation inspection in the future. Among them, mobile edge computing provides a promising architecture to meet the explosive growth of communication and computing needs of inspection robot. Inspection robot can transmit the collected High Definition (HD) video to adjacent edge servers for data processing and state research and judgment. However, the communication constraints of long-distance transmission, high reliability and low delay pose challenges to task offloading optimization. Therefore, this paper introduced Unmanned Aerial Vehicle (UAV) and established UAV assisted mobile edge computing system. UAV assisted and mobile edge computing are combined to form edge computing nodes. In this way, it provided communication and computing services to the IR for fast data processing. Specifically, in order to optimize the system energy consumption, a resource allocation strategy based on genetic algorithm is proposed. By optimizing the offloading decision and computing resource allocation of the IRs, the computing task of the IRs are offloaded to an energy-efficient UAV. The experimental results show that the resource allocation strategy based on genetic algorithm can effectively reduce the energy consumption and cost of UAVs and IRs, and effectively realize the reasonable allocation of resources. The results verify the effectiveness and reliability of the algorithm in the real scene.
{"title":"Resource allocation in UAV assisted air ground intelligent inspection system","authors":"Zhuoya Zhang , Fei Xu , Zengshi Qin , Yue Xie","doi":"10.1016/j.cogr.2021.12.002","DOIUrl":"10.1016/j.cogr.2021.12.002","url":null,"abstract":"<div><p>With the progress of power grid technology and intelligent technology, intelligent inspection robot (IR) came into being and are expected to become the main force of substation inspection in the future. Among them, mobile edge computing provides a promising architecture to meet the explosive growth of communication and computing needs of inspection robot. Inspection robot can transmit the collected High Definition (HD) video to adjacent edge servers for data processing and state research and judgment. However, the communication constraints of long-distance transmission, high reliability and low delay pose challenges to task offloading optimization. Therefore, this paper introduced Unmanned Aerial Vehicle (UAV) and established UAV assisted mobile edge computing system. UAV assisted and mobile edge computing are combined to form edge computing nodes. In this way, it provided communication and computing services to the IR for fast data processing. Specifically, in order to optimize the system energy consumption, a resource allocation strategy based on genetic algorithm is proposed. By optimizing the offloading decision and computing resource allocation of the IRs, the computing task of the IRs are offloaded to an energy-efficient UAV. The experimental results show that the resource allocation strategy based on genetic algorithm can effectively reduce the energy consumption and cost of UAVs and IRs, and effectively realize the reasonable allocation of resources. The results verify the effectiveness and reliability of the algorithm in the real scene.</p></div>","PeriodicalId":100288,"journal":{"name":"Cognitive Robotics","volume":"2 ","pages":"Pages 1-12"},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2667241321000215/pdfft?md5=52655729279f3a497faeb732baa533df&pid=1-s2.0-S2667241321000215-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80206059","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-01-01DOI: 10.1016/j.cogr.2022.08.003
Shuhua Bai , Xiaojian Qiu , Rongqun Hu , Yunqiang Wu
Positron emission tomography (PET) and computed tomography (CT) scanner image analysis plays an important role in clinical radiotherapy treatment. PET and CT images provide complementary cues for identifying tumor tissues. In specific, PET images can clearly denote the tumor tissue, whereas this source suffers from the problem of low spatial resolution. On the contrary, CT images have a high resolution, but they can not recognize the tumor from normal tissues. In this work, we firstly fuse PET and CT images by using the guided filter. Then, a region and edge-based level set model is proposed to segment PET-CT fusion images. At last, a normalization term is designed by combining length, distance and H1 terms with the aim to improve segmentation accuracy. The proposed method was validated in the robust delineation of lung tumor tissues on 20 PET-CT samples. Both qualitative and quantitative results demonstrate significant improvement compared to both the data-independent and deep learning based segmentation methods.
{"title":"A novel level set model initialized with guided filter for automated PET-CT image segmentation","authors":"Shuhua Bai , Xiaojian Qiu , Rongqun Hu , Yunqiang Wu","doi":"10.1016/j.cogr.2022.08.003","DOIUrl":"10.1016/j.cogr.2022.08.003","url":null,"abstract":"<div><p>Positron emission tomography (PET) and computed tomography (CT) scanner image analysis plays an important role in clinical radiotherapy treatment. PET and CT images provide complementary cues for identifying tumor tissues. In specific, PET images can clearly denote the tumor tissue, whereas this source suffers from the problem of low spatial resolution. On the contrary, CT images have a high resolution, but they can not recognize the tumor from normal tissues. In this work, we firstly fuse PET and CT images by using the guided filter. Then, a region and edge-based level set model is proposed to segment PET-CT fusion images. At last, a normalization term is designed by combining length, distance and H<sup>1</sup> terms with the aim to improve segmentation accuracy. The proposed method was validated in the robust delineation of lung tumor tissues on 20 PET-CT samples. Both qualitative and quantitative results demonstrate significant improvement compared to both the data-independent and deep learning based segmentation methods.</p></div>","PeriodicalId":100288,"journal":{"name":"Cognitive Robotics","volume":"2 ","pages":"Pages 193-201"},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2667241322000180/pdfft?md5=19e625c37228b4881aaccfb4c3123000&pid=1-s2.0-S2667241322000180-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80496687","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-01-01DOI: 10.1016/j.cogr.2022.06.003
Ke Li , Zhonghua Luo , Tong Zhang , Yinglan Ruan , Dan Zhou
Extracting salient regions plays an important role in computer vision tasks, e.g., object detection, recognition and video compression. Previous saliency detection study is mostly conducted on individual frames and tends to extract saliency with spatial cues. The development of various motion feature further extends the saliency concept to the motion saliency from videos. In contrast to image-based saliency extraction, video-based saliency extraction is more challenging due to the complicated distractors, e.g., the background dynamics and shadows. In this paper, we propose a novel saliency extraction method by fusing temporal and spatial cues. In specific, the long-term and short-term variations are comprehensively fused to extract the temporal cue, which is then utilized to establish the background guidance for generating the spatial cue. Herein, the long-term variations and spatial cues jointly highlight the contrast between objects and the background, which can solve the problem caused by shadows. The short-term variations contribute to the removal of background dynamics. Spatiotemporal cues are fully exploited to constrain the saliency extraction across frames. The saliency extraction performance of our method is demonstrated by comparing it to both unsupervised and supervised methods. Moreover, this novel saliency extraction model is applied in the video compression tasks, helping to accelerate the video compression task and achieve a larger PSNR value for the region of interest (ROI).
{"title":"Spatiotemporal cue fusion-based saliency extraction and its application in video compression","authors":"Ke Li , Zhonghua Luo , Tong Zhang , Yinglan Ruan , Dan Zhou","doi":"10.1016/j.cogr.2022.06.003","DOIUrl":"10.1016/j.cogr.2022.06.003","url":null,"abstract":"<div><p>Extracting salient regions plays an important role in computer vision tasks, e.g., object detection, recognition and video compression. Previous saliency detection study is mostly conducted on individual frames and tends to extract saliency with spatial cues. The development of various motion feature further extends the saliency concept to the motion saliency from videos. In contrast to image-based saliency extraction, video-based saliency extraction is more challenging due to the complicated distractors, e.g., the background dynamics and shadows. In this paper, we propose a novel saliency extraction method by fusing temporal and spatial cues. In specific, the long-term and short-term variations are comprehensively fused to extract the temporal cue, which is then utilized to establish the background guidance for generating the spatial cue. Herein, the long-term variations and spatial cues jointly highlight the contrast between objects and the background, which can solve the problem caused by shadows. The short-term variations contribute to the removal of background dynamics. Spatiotemporal cues are fully exploited to constrain the saliency extraction across frames. The saliency extraction performance of our method is demonstrated by comparing it to both unsupervised and supervised methods. Moreover, this novel saliency extraction model is applied in the video compression tasks, helping to accelerate the video compression task and achieve a larger PSNR value for the region of interest (ROI).</p></div>","PeriodicalId":100288,"journal":{"name":"Cognitive Robotics","volume":"2 ","pages":"Pages 177-185"},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2667241322000131/pdfft?md5=181cb8030eca6d4778b64500c49f1fa8&pid=1-s2.0-S2667241322000131-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76038010","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-01-01DOI: 10.1016/j.cogr.2022.06.002
Fan Linjuan, Sun Yongyong, Xu Fei, Zhou Hnghang
In view of the current knowledge graph embedding, it mainly focuses on symmetry/opposition, inversion and combination of relationship patterns, and does not fully consider the structure of the knowledge graph. We propose a Knowledge Graph Embedding Based on Semantic Hierarchy (SHKE), which fully considers the information of knowledge graph by fusing the semantic information of the knowledge graph and the hierarchical information. The knowledge graph is mapped to a polar coordinate system, where concentric circles naturally reflect the hierarchy, and entities can be divided into modulus parts and phase parts, and then the modulus part of the polar coordinate system is mapped to the relational vector space through the relational vector, thus the modulus part takes into account the semantic information of the knowledge graph, and the phase part takes into account the hierarchical information. Experiments show that compared with other models, the proposed model improves the knowledge graph link prediction index Hits@10% by about 10% and the accuracy of the triple group classification experiment by about 10%.
{"title":"Knowledge graph embedding based on semantic hierarchy","authors":"Fan Linjuan, Sun Yongyong, Xu Fei, Zhou Hnghang","doi":"10.1016/j.cogr.2022.06.002","DOIUrl":"10.1016/j.cogr.2022.06.002","url":null,"abstract":"<div><p>In view of the current knowledge graph embedding, it mainly focuses on symmetry/opposition, inversion and combination of relationship patterns, and does not fully consider the structure of the knowledge graph. We propose a Knowledge Graph Embedding Based on Semantic Hierarchy (SHKE), which fully considers the information of knowledge graph by fusing the semantic information of the knowledge graph and the hierarchical information. The knowledge graph is mapped to a polar coordinate system, where concentric circles naturally reflect the hierarchy, and entities can be divided into modulus parts and phase parts, and then the modulus part of the polar coordinate system is mapped to the relational vector space through the relational vector, thus the modulus part takes into account the semantic information of the knowledge graph, and the phase part takes into account the hierarchical information. Experiments show that compared with other models, the proposed model improves the knowledge graph link prediction index Hits@10% by about 10% and the accuracy of the triple group classification experiment by about 10%.</p></div>","PeriodicalId":100288,"journal":{"name":"Cognitive Robotics","volume":"2 ","pages":"Pages 147-154"},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S266724132200012X/pdfft?md5=eff502f209037b9c55f942f433d918f1&pid=1-s2.0-S266724132200012X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83608118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}