Automated diagnosis of chest X-rays is pivotal in radiology, aiming to alleviate the workload of radiologists. Traditional methods primarily rely on visual features or label dependence, which is a limitation in detecting nuanced or rare lesions. To address this, we present KEXNet, a pioneering knowledge-enhanced X-ray lesion detection model. KEXNet employs a unique strategy akin to expert radiologists, integrating a knowledge graph based on expert annotations with an interpretable graph learning approach. This novel method combines object detection with a graph neural network, facilitating precise local lesion detection. For global lesion detection, KEXNet synergizes knowledge-enhanced local features with global image features, enhancing diagnostic accuracy. Our evaluations on three benchmark datasets demonstrate that KEXNet outshines existing models, particularly in identifying small or infrequent lesions. Notably, on the Chest ImaGenome dataset, KEXNet’s AUC for local lesion detection surpasses 8.9% compared to the state-of-the-art method AnaXNet, showcasing its potential in revolutionizing automated chest X-ray diagnostics.