Humongous amounts of data bring various challenges to face image retrieval. This paper proposes an efficient method to solve those problems. Firstly, we use accurate facial landmark locations as shape features. Secondly, we utilise shape priors to provide discriminative texture features for convolutional neural networks. These shape and texture features are fused to make the learned representation more robust. Finally, in order to increase efficiency, a coarse-to-fine search mechanism is exploited to efficiently find similar objects. Extensive experiments on the CASIA-WebFace, MSRA-CFW, and LFW datasets illustrate the superiority of our method.