Recent advances in drone-view object detection

Abstract

在人工智能技术的支持下,无人机初步获得智能感知能力,在实际应用中展现出高效灵活的数据收集能力。无人机视角下的目标检测作为关键核心技术,在诸多领域中发挥着不可替代的作用,具有重要的研究意义。为了进一步展现无人机视角下的目标检测研究进展,本文对无人机视角下的目标检测算法进行了全面总结,并对已有算法进行了归类、分析和比较。1)介绍无人机视角下的目标检测概念,并总结无人机视角下目标检测所面临的目标尺度、空间分布、样本数量、类别语义以及优化目标等 5 大不均衡挑战。在介绍现有研究方法的基础上,特别整理并介绍了无人机视角下目标检测算法在交通监控、电力巡检、作物分析和灾害救援等实际场景中的应用。2)重点阐述从数据增强策略、多尺度特征融合、区域聚焦策略、多任务学习以及模型轻量化等方面提升无人机视角下目标检测性能的方法,总结这些方法的优缺点并分析了其与现存挑战之间的关联性。3)全面介绍基于无人机视角的目标检测数据集,并呈现已有算法在两个较常用公共数据集上的性能评估。4)对无人机视角下目标检测技术的未来发展方向进行了展望。;Given the support of artificial intelligence technology, drones have initially acquired intelligent sensing capabilities and have demonstrated efficient and flexible data collection in practical applications.Drone-view object detection, which aims to locate specific objects in aerial images, plays an irreplaceable role in many fields and has important research significance.For example, drones with highly mobile and flexible deployment have remarkable advantages in accident handling, order management, traffic guidance, and flow detection, making them irreplaceable in traffic monitoring.As for disaster emergency rescue, drones with aerial vision and high mobility can achieve efficient search and safe rescue in large areas, locate people quickly and accurately in distress, and help rescuers control the situation, thereby ensuring the safety of people in distress.This study provides a comprehensive summary of the challenges in object detection based on the unmanned aerial vehicle(UAV)perspective to portray further the development of drone-view object detection.The existing algorithms and related datasets are also introduced.First, this study briefly introduces the concept of object detection in drone view and summarizes the five imbalance challenges in object detection in drone view, such as scale imbalance, spatial imbalance, class imbalance, semantic imbalance, and objective imbalance.This study analyzes and summarizes the challenges of drone-view object detection based on the aforementioned imbalances by using quantitative data analysis and visual qualitative analysis.1)Object scale imbalance is the most focused challenge in current research.It comes from the unique aerial view of drones.The changes in the drone's height and angle bring drastic changes to the object scale in the acquired images.The distance of the lens from the photographed object under the drone view is often far.This scenario results in numerous small objects in the image and makes capturing useful features for object detection difficult for the existing detectors.2)Different regions of drone-view images have great differences, and most objects are concentrated in the minor area of images, i.e., the spatial distribution of objects is enormously uneven.On the one hand, the clustering of dense objects in small areas generates occlusion.The detection model needs to devote considerable attention to this occlusion to distinguish different objects effectively.On the other hand, treating equally different areas wastes many computational resources in vanilla areas, limiting the improvement of object detection performance.3)The problem of class imbalance in the drone view is divided into two categories.One is the positive-negative sample imbalance problem caused by the gap between the front and rear views shared in the image.The other is the imbalanced numbers of different categories caused by the number of samples in the real world.4)The semantic pieces of information defined by different category labels in the drone-view object detection dataset are often similar, resulting in only subtle differences between different categories.However, significantly different representations of objects exist in the same category, which together form the semantic imbalance problem.5)Drone-view object detection often faces the problem of unbalanced optimization targets, i.e., the contradiction between the high computational demand for high-resolution images and the limited computing power of low-power chips is difficult to balance.These unbalanced problems bring enormous challenges to object detection from the UAV viewpoint.However, even the most advanced object detection algorithms currently available can hardly achieve an average accuracy rate of 40% on aerial images, which is far below the performance of general object detection tasks.Therefore, many scholars have conducted many studies.These research methods can be summarized as optimization ideas to solve these imbalance problems.In this study, we collect relevant research works, which are sorted and analyzed according to the countries of authors, institutions, published journals or conferences, years, the category of methods, and the solved problem.The present study presents the challenging problems solved by previous research and the development trends of existing methods.This study also focuses on the methods of improving drone-view object detection performance in terms of data augmentation, multiscale feature fusion, region searching strategies, multitask learning, and lightweight model.The advantages and disadvantages of these methods for different problems are systematically summarized and analyzed.Besides introducing existing methods, the present study compiles and introduces the applications of drone-view object detection in practical scenarios, such as traffic monitoring, power inspection, crop analysis, and disaster rescue.These applications further emphasize the significance of object detection in drone view.Then, this study collects and organizes UAV datasets suitable for object detection tasks.These datasets are present from various perspectives, such as year, published journals or conferences, annotation information, and number of citations.In particular, the present study provides the performance evaluation of the existing algorithms on two commonly used public datasets.The presentation of these performance data is expected to help researchers understand the current state of development of drone-view object detection and promote further development in this field.Finally, this study provides an outlook on the future direction of drone-view object detection by considering the aforementioned imbalance problems.The promising research includes the following:1)data augmentation:providing the network with enough high-quality learning samples by considering the specific characteristics of drone-view images based on the conventional data augmentation strategy is a good idea;2)multiscale representation:how to avoid the interference of background noise in feature fusion and effectively extract information at different scales using an efficient fusion strategy is an urgent problem to be solved;3)visual inference:using information unique to the viewpoint of drones, mining contextual information from images to facilitate image recognition, and using easy-to-detect objects to improve the performance of difficult-to-detect objects are directions worthy of deep consideration.