View context based 2D sketch-3D model alignment

Abstract

2D sketch-3D model alignment is important for many applications such as sketch-based 3D model retrieval, sketch-based 3D modeling as well as model-based vision and recognition. In this paper, we propose a 2D sketch-3D model alignment algorithm using view context and shape context matching. A sketch consists of a set of curves. A 3D model is typically a 3D triangle mesh. It includes two main steps: precomputation and actual alignment. In the precomputation, we extract the view context features of a set of sample views for a 3D model to be aligned. To speed up the precomputation, two computationally efficient and rotation-invariant features, Zernike moments and Fourier descriptors are used to represent a view. In the actual alignment, we prune most sample views which are dissimilar to the sketch very quickly based on their view context similarities. Finally, to find an approximate pose, we only compare the sketch with a very small portion (e.g. 5% in our experiments) of the sample views based on shape context matching. Experiments on two types of datasets show that the algorithm can align 2D sketches with 3D models approximately.