View-based 3-D object retrieval has become an emerging topic in recent years, especially with the fast development of visual content acquisition devices, such as mobile phones with cameras. Extensive research efforts have been dedicated to this task, while it is still difficult to measure the relevance between two objects with multiple views. In recent years, learning-based methods have been investigated in view-based 3-D object retrieval, such as graph-based learning. It is noted that the graph-based methods suffer from the high computational cost from the graph construction and the corresponding learning process. In this paper, we introduce a general framework to accelerate the learning-based view-based 3-D object matching in large scale data. Given a query object $Q$ and one object $O$ from a 3-D dataset $𝒟$, the first step is to extract a small set of candidate relevant 3-D objects for object $O$. Then multiple hypergraphs can be constructed based on this small set of 3-D objects and the learning on the fused hypergraph is conducted to generate the relevance between $Q$ and $O$, which can be further used in the retrieval procedure. Experiments demonstrate the effectiveness of the proposed framework.