Rice (Oryza sativa) provides a staple food source for more than half the world population. However, the current pace of rice breeding in yield growth is insufficient to meet the food demand of the ever-increasing global population. Genomic selection (GS) holds a great potential to accelerate breeding progress and is cost-effective via early selection before phenotypes are measured. Previous simulation and experimental studies have demonstrated the usefulness of GS in rice breeding. However, several affecting factors and limitations require careful consideration when performing GS. In this review, we summarize the major genetics and statistical factors affecting predictive performance as well as current progress in the application of GS to rice breeding. We also highlight effective strategies to increase the predictive ability of various models, including GS models incorporating functional markers, genotype by environment interactions, multiple traits, selection index, and multiple omic data. Finally, we envision that integrating GS with other advanced breeding technologies such as unmanned aerial vehicles and open-source breeding platforms will further improve the efficiency and reduce the cost of breeding.