To obtain a suitable scheduling scheme in an effective time range, the minimum completion time is taken as the objective of Flexible Job Shop scheduling Problems (FJSP) with different scales, and Composite Dispatching Rules (CDRs) are applied to generate feasible solutions. Firstly, the binary tree coding method is adopted, and the constructed function set is normalized. Secondly, a CDR mining approach based on an Improved Genetic Programming Algorithm (IGPA) is designed. Two population initialization methods are introduced to enrich the initial population, and a superior and inferior population separation strategy is designed to improve the global search ability of the algorithm. At the same time, two individual mutation methods are introduced to improve the algorithm’s local search ability, to achieve the balance between global search and local search. In addition, the effectiveness of the IGPA and the superiority of CDRs are verified through comparative analysis. Finally, Deep Reinforcement Learning (DRL) is employed to solve the FJSP by incorporating the CDRs as the action set, the selection times are counted to further verify the superiority of CDRs.