The steel plate is one of the main products in steel industries, and its surface quality directly affects the final product performance. How to detect surface defects of steel plates in real time during the production process is a challenging problem. The single or fixed model compression method cannot be directly applied to the detection of steel surface defects, because it is difficult to consider the diversity of production tasks, the uncertainty caused by environmental factors, such as communication networks, and the influence of process and working conditions in steel plate production. In this paper, we propose an adaptive model compression method for steel surface defect online detection based on expert knowledge and working conditions. First, we establish an expert system to give lightweight model parameters based on the correlation between defect types and manufacturing processes. Then, lightweight model parameters are adaptively adjusted according to working conditions, which improves detection accuracy while ensuring real-time performance. The experimental results show that compared with the detection method of constant lightweight parameter model, the proposed method makes the total detection time cut down by 23.1%, and the deadline satisfaction ratio increased by 36.5%, while upgrading the accuracy by 4.2% and reducing the false detection rate by 4.3%.