A CNN-based QTMT partitioning decision for the VVC standard

The Versatile Video Coding (VVC) standard enhances the coding performance in terms of bitrate and quality compared to its predecessor High Efficiency Video Coding (HEVC) at the expense of an additional coding complexity. This complexity is due to the new partitioning improvement named quadtree with nested multi-type tree (QTMT) which includes more partition shapes increasing the rate-distortion optimization (RDO) complexity. In this paper, a fast QTMT partition approach based on the Convolutional Neural Network (CNN) is proposed for the purpose of reducing the computational complexity. In fact, an intra QTMT decision tree method using a Convolutional Neural Network-binary tree horizontal (CNN-BTH) model is elaborated to determine the BTH decision depths at the 32×32 block size. The suggested fast partitioning algorithm achieves an important gain in the encoding time up to 64.26% compared to the original VVC software reference VTM-3.0. It allows a significant computational complexity reduction of 58.28% on average and an acceptable loss in the encoding performance.