Deciding the penalty of a law case has always been a complex process, which may involve with much coordination. Despite the judicial study based on the rules and conditions, artificial intelligence and machine learning has rarely been used to study the problem of penalty inferring, leaving the large amount of law cases as well as various factors among them untouched. This paper aims to incorporate the state-of-the-art artificial intelligence methods to exploit to what extent this problem can be alleviated. We first analyze 145000 law cases and observe that there are two sorts of labels, temporal labels and spatial labels, which have unique characteristics. Temporal labels and spatial labels tend to converge towards the final penalty, on condition that the cases are of the same category. In light of this, we propose a latent-class probabilistic generative model, namely Penalty Topic Model (PTM), to infer the topic of law cases, and the temporal and spatial patterns of topics embedded in the case judgment. Then, the learnt knowledge is utilized to automatically cluster all cases accordingly in a unified way. We conduct extensive experiments to evaluate the performance of the proposed PTM on a real large-scale dataset of law cases. The experimental results show the superiority of our proposed PTM.