Ultra-FastNet: an end-to-end learnable network for multi-person posture prediction

Abstract

At present, the top-down approach requires the introduction of pedestrian detection algorithms in multi-person pose estimation. In this paper, we propose an end-to-end trainable human pose estimation network named Ultra-FastNet, which has three main components: shape knowledge extractor, corner prediction module, and human body geometric knowledge encoder. Firstly, the shape knowledge extractor is built using the ultralightweight bottleneck module, which effectively reduces network parameters and effectively learns high-resolution local representations of keypoints; the global attention module was introduced to build an ultralightweight bottleneck block to capture keypoint shape knowledge and build high-resolution features. Secondly, the human body geometric knowledge encoder, which is made up of Transformer, was introduced to modeling and discovering body geometric knowledge in data. The network uses both shape knowledge and body geometric knowledge which is called knowledge-enhanced, to deduce keypoints. Finally, the pedestrian detection task is modeled as a keypoint detection task using the corner prediction module. As a result, an end-to-end multitask network can be created without the requirement to include pedestrian detection algorithms in order to execute multi-person pose estimation. In the experiments, we show that Ultra-FastNet can achieve high accuracy on the COCO2017 and MPII datasets. Furthermore, experiments show that our method outperforms the mainstream lightweight network.