Workload prediction is critical in enabling proactive resource management of cloud applications. Accurate workload prediction is valuable for cloud users and providers as it can effectively guide many practices, such as performance assurance, cost reduction, and energy consumption optimization. However, cloud workload prediction is highly challenging due to the complexity and dynamics of workloads, and various solutions have been proposed to enhance the prediction behavior. This paper aims to provide an in-depth understanding and categorization of existing solutions through extensive literature reviews. Unlike existing surveys, for the first time, we comprehensively sort out and analyze the development landscape of workload prediction from a new perspective, i.e., application-oriented rather than prediction methodologies per se. Specifically, we first introduce the basic features of workload prediction, and then analyze and categorize existing efforts based on two significant characteristics of cloud applications: variability and heterogeneity. Furthermore, we also investigate how workload prediction is applied to resource management. Finally, open research opportunities in workload prediction are highlighted to foster further advancements.