Adaptive Multi-Level Saliency Network in 3D Generation

Abstract

Many CNNs with encoder-decoder structure are widely used in supervised 3D voxel generation. However, their convolutional encoders are usually too simple which causes some local features to degrade during convolution, so it is difficult to extract a good feature representation from an input image by using a simple encoder. Some CNNs apply skip-connection layer for the encoder to reduce the degradation, but general skip-connection layer such as residual layer is not good enough especially when the quantity of convolutional layers in the encoder is relatively small. In this paper, we propose a novel structure called adaptive multi-level saliency network (AMSN) to reduce the degradation of local features. The major innovations of AMSN are multi-level saliency convolution kernels (MSCK) and saliency fusion layer. Different from the kernels used in general skip-connection layer, MSCK are adaptively learned from multi-level salient feature maps rather than initialized randomly. The salient feature maps are sampled from multiple layers in the encoder. MSCK can acquire multi-level features more easily so that we utilize MSCK to acquire local features from low-level layer before the degradation. After that, the acquired local features are fused back into encoder through a saliency fusion layer to reduce the degradation. We evaluated our approach on the ShapeNet and ModelNet40 dataset. The results indicate that our AMSN performs better than related works.