{"title":"Enhancing robustness in asynchronous feature tracking for event cameras through fusing frame steams","authors":"Haidong Xu, Shumei Yu, Shizhao Jin, Rongchuan Sun, Guodong Chen, Lining Sun","doi":"10.1007/s40747-024-01513-0","DOIUrl":null,"url":null,"abstract":"<p>Event cameras produce asynchronous discrete outputs due to the independent response of camera pixels to changes in brightness. The asynchronous and discrete nature of event data facilitate the tracking of prolonged feature trajectories. Nonetheless, this necessitates the adaptation of feature tracking techniques to efficiently process this type of data. In addressing this challenge, we proposed a hybrid data-driven feature tracking method that utilizes data from both event cameras and frame-based cameras to track features asynchronously. It mainly includes patch initialization, patch optimization, and patch association modules. In the patch initialization module, FAST corners are detected in frame images, providing points responsive to local brightness changes. The patch association module introduces a nearest-neighbor (NN) algorithm to filter new feature points effectively. The patch optimization module assesses optimization quality for tracking quality monitoring. We evaluate the tracking accuracy and robustness of our method using public and self-collected datasets, focusing on average tracking error and feature age. In contrast to the event-based Kanade–Lucas–Tomasi tracker method, our method decreases the average tracking error ranging from 1.3 to 29.2% and boosts the feature age ranging from 9.6 to 32.1%, while ensuring the computational efficiency improvement of 1.2–7.6%. Thus, our proposed feature tracking method utilizes the unique characteristics of event cameras and traditional cameras to deliver a robust and efficient tracking system.</p>","PeriodicalId":10524,"journal":{"name":"Complex & Intelligent Systems","volume":null,"pages":null},"PeriodicalIF":5.0000,"publicationDate":"2024-06-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Complex & Intelligent Systems","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s40747-024-01513-0","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Event cameras produce asynchronous discrete outputs due to the independent response of camera pixels to changes in brightness. The asynchronous and discrete nature of event data facilitate the tracking of prolonged feature trajectories. Nonetheless, this necessitates the adaptation of feature tracking techniques to efficiently process this type of data. In addressing this challenge, we proposed a hybrid data-driven feature tracking method that utilizes data from both event cameras and frame-based cameras to track features asynchronously. It mainly includes patch initialization, patch optimization, and patch association modules. In the patch initialization module, FAST corners are detected in frame images, providing points responsive to local brightness changes. The patch association module introduces a nearest-neighbor (NN) algorithm to filter new feature points effectively. The patch optimization module assesses optimization quality for tracking quality monitoring. We evaluate the tracking accuracy and robustness of our method using public and self-collected datasets, focusing on average tracking error and feature age. In contrast to the event-based Kanade–Lucas–Tomasi tracker method, our method decreases the average tracking error ranging from 1.3 to 29.2% and boosts the feature age ranging from 9.6 to 32.1%, while ensuring the computational efficiency improvement of 1.2–7.6%. Thus, our proposed feature tracking method utilizes the unique characteristics of event cameras and traditional cameras to deliver a robust and efficient tracking system.
期刊介绍:
Complex & Intelligent Systems aims to provide a forum for presenting and discussing novel approaches, tools and techniques meant for attaining a cross-fertilization between the broad fields of complex systems, computational simulation, and intelligent analytics and visualization. The transdisciplinary research that the journal focuses on will expand the boundaries of our understanding by investigating the principles and processes that underlie many of the most profound problems facing society today.