SeaConvNeXt: A Lightweight Two-Branch Network Architecture for Efficient Prediction of Specific IHC Proteins and Antigens on Hematoxylin and Eosin (H&E) Images

Immunohistochemistry (IHC) is a vital technique for detecting specific proteins and antigens in tissue sections using antibodies, aiding in the analysis of tumor growth and metastasis. However, IHC is costly and time-consuming, making it challenging to implement on a large scale. To address this issue, we introduce a method that enables virtual IHC staining directly on Hematoxylin and Eosin (H&E) images. Firstly, we have developed a novel registration technique, called Bi-stage Registration based on density Clustering (BiReC), to enhance the registration efficiency between H&E and IHC images. This method involves automatically generating numerous Regions Of Interest (ROI) labels on the H&E image for model training, with the labels being determined by the intensity of IHC staining. Secondly, we propose a novel two-branch network architecture, called SeaConvNeXt, which integrates a lightweight Squeeze-Enhanced Axial (SEA) attention mechanism to efficiently extract and fuse multi-level local and global features from H&E images for direct prediction of specific proteins and antigens. The SeaConvNeXt consists of a ConvNeXt branch and a global fusion branch. The ConvNeXt branch extracts multi-level local features at four stages, while the global fusion branch, including an SEA Transformer module and three global blocks, is designed for global feature extraction and multiple feature fusion. Our experiments demonstrate that SeaConvNeXt outperforms current state-of-theart methods on two public datasets with corresponding IHC and H&E images, achieving an AUC of 90.7% on the HER2SC dataset and 82.5% on the CRC dataset. These results suggest that SeaConvNeXt has great potential for predicting virtual IHC staining on H&E images.