RGB-D salient object detection: A survey

Tao Zhou; Deng-Ping Fan; Ming-Ming Cheng; Jianbing Shen; Ling Shao

doi:10.1007/s41095-020-0199-z

| Sign up

PDF (13.3 MB)

Cite

EndNote(RIS) BibTeX

Collect

Submit Manuscript

Show Outline

Figures (14)

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Tables (11)

Table 1

Table 2

Table 3

Table 4

Table 5

Review Article | Open Access

RGB-D salient object detection: A survey

Tao Zhou^¹, Deng-Ping Fan^¹

(), Ming-Ming Cheng^², Jianbing Shen^¹, Ling Shao^¹

1Inception Institute of Artificial Intelligence (IIAI), Abu Dhabi,United Arab Emirates

2CS, Nankai University, Tianjin 300350, China

Show Author Information

Abstract

Salient object detection, which simulateshuman visual perception in locating the most significant object(s) in a scene, has been widely applied to various computer vision tasks. Now, the advent of depth sensors means that depth maps can easily be captured; this additional spatial information can boost the performance of salient object detection. Although various RGB-D based salient object detection models with promising performance have been proposed over the past several years, an in-depth understanding of these models and the challenges in this field remains lacking. In this paper, we provide a comprehensive survey of RGB-D based salient object detection models from various perspectives, and review related benchmark datasets in detail. Further, as light fields can also provide depth maps, we review salient object detection models and popular benchmark datasets from this domain too. Moreover, to investigate the ability of existing models to detect salient objects, we have carried out a comprehensive attribute-based evaluation of several representative RGB-D based salient object detection models. Finally, we discuss several challenges and open directions of RGB-D based salient object detection for future research. All collected models, benchmark datasets, datasets constructed for attribute-based evaluation, and related code are publicly available at https://github.com/taozh2017/RGBD-SODsurvey.

Keywords

RGB-D saliency light fields benchmarks

References

[1]

Fan,

D. P.

; Cheng,

M. M.

; Liu,

J. J.

; Gao,

S. H.

; Hou,

Q. B.

; Borji,

Salient objects in clutter: Bringing salient object detection to the foreground. In: Computer Vision - ECCV 2018. Lecture Notes in Computer Science, Vol. 11219. Ferrari,

; Hebert,

; Sminchisescu,

; Weiss,

Eds. Springer Cham, 196-212, 2018.

[2]

Nie,

G.-Y.

; Cheng,

M.-M.

; Liu,

; Liang,

; Fan,

D.-P.

; Liu,

; Wang,

Multi-level context ultra-aggregation for stereo matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3278-3286, 2019.

Crossref

[3]

Zhu,

J. Y.

; Wu,

J. J.

; Xu,

; Chang,

, Tu,

Z. W.

Unsupervised object class discovery via saliency-guided multiple class learning. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 37, No. 4, 862-875, 2015.

Crossref Google Scholar

[4]

Fan,

D. P.

; Li,

T. P.

; Lin,

; Ji,

G. P.

; Zhang,

D. W.

; Cheng,

M. M.

; Fu,

; Shen,

Re-thinking co-salient object detection. arXiv preprint arXiv:2007.03380, 2020.

Google Scholar

[5]

Rapantzikos,

; Avrithis,

; Kollias,

Dense saliency-based spatiotemporal feature points for action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1454-1461, 2009.

Crossref

[6]

Fan,

D.-P.

; Wang,

; Cheng,

M.-M.

; Shen,

Shifting more attention to video salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8554-8564, 2019.

Crossref

[7]

Wang,

W. G.

; Shen,

J. B.

; Yang,

R. G.

; Porikli,

Saliency-aware video object segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 40, No. 1, 20-33, 2018.

Crossref Google Scholar

[8]

Song,

H. M.

; Wang,

W. G.

; Zhao,

S. Y.

; Shen,

J. B.

; Lam,

K. M.

Pyramid dilated deeper ConvLSTM for video salient object detection. In: Computer Vision - ECCV 2018. Lecture Notes in Computer Science, Vol. 11215. Ferrari,

; Hebert,

; Sminchisescu,

; Weiss,

Eds. Springer Cham, 744-760, 2018.

[9]

Wang,

W. G.

; Shen,

J. B.

; Shao,

Video salient object detection via fully convolutional networks. IEEE Transactions on Image Processing Vol. 27, No. 1, 38-49, 2018.

Crossref Google Scholar

[10]

Shimoda,

; Yanai,

Distinct class-specific saliency maps for weakly supervised semantic segmentation. In: Computer Vision - ECCV 2016. Lecture Notes in Computer Science, Vol. 9908. Leibe,

; Matas,

; Sebe,

; Welling,

Eds. Springer Cham, 218-234, 2016.

[11]

Zeng,

; Zhuge,

; Lu,

; Zhang,

Joint learning of saliency detection and weakly supervised semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, 7223-7233, 2019.

[12]

Fan,

D. P.

; Ji,

G. P.

; Zhou,

; Chen,

; Fu,

H. Z.

; Shen,

J. B.

; Shao,

PraNet: Parallel reverse attention network for polyp segmentation. In: Medical Image Computing and Computer Assisted Intervention - MICCAI 2020. Lecture Notes in Computer Science, Vol. 12266. Martel,

A. L.

et al. Eds. Springer Cham, 263-273, 2020.

Crossref

[13]

Fan,

D. P.

; Zhou,

; Ji,

G. P.

; Zhou,

; Chen,

; Fu,

H. Z.

; Shen,

; Shao,

Inf-Net: Automatic COVID-19 lung infection segmentation from CT images. IEEE Transactions on Medical Imaging Vol. 39, No. 8, 2626-2637, 2020.

Crossref Google Scholar

[14]

Wu,

Y.-H.

; Gao,

S.-H.

; Mei,

; Xu,

; Fan,

D.-P.

; Zhao,

C.- W.

; Cheng,

M.-M.

JCS: An explainable COVID-19 diagnosis system by joint classification and segmentation. arXiv preprint arXiv:2004.07054, 2020.

Google Scholar

[15]

Mahadevan,

; Vasconcelos,

Saliency-based discriminant tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1007-1013, 2009.

Crossref

[16]

Hong,

; You,

; Kwak,

; Han,

Online tracking by learning discriminative saliency map with convolutional neural network. In: Proceedings of the International Conference on Machine Learning, 597-606, 2015.

[17]

Zhao,

; Oyang,

; Wang,

X. G.

Person re-identification by saliency learning. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 2, 356-370, 2017.

Crossref Google Scholar

[18]

Martinel,

, Micheloni,

, Foresti,

G. L.

Kernelized saliency-based person Re-identification through multiple metric learning. IEEE Transactions on Image Processing Vol. 24, No. 12, 5645-5658, 2015.

Crossref Google Scholar

[19]

Fan,

D.-P.

; Ji,

G.-P.

; Sun,

; Cheng,

M.-M.

; Shen,

; Shao,

Camouaged object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2777-2787, 2020.

Crossref

[20]

Liu,

; Fan,

A model of visual attention for natural image retrieval. In: Proceedings of the IEEE Conference on Information Science and Cloud Computing Companion, 728-733, 2013.

Crossref

[21]

Zhao,

J.-X.

; Liu,

J.-J.

; Fan,

D.-P.

; Cao,

; Yang,

; Cheng,

M.-M.

EGNet: Edge guidance network for salient object detection. In: Proceedings of the IEEE International Conference on Computer Vision, 8779-8788, 2019.

Crossref

[22]

Tu,

W.-C.

; He,

; Yang,

; Chien,

S.-Y.

Real-time salient object detection with a minimum spanning tree. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2334-2342, 2016.

[23]

Xia,

; Li,

; Chen,

; Zheng,

; Zhang,

What is and what is not a salient object? Learning salient object detector by ensembling linear exemplar regressors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4142-4150, 2017.

Crossref

[24]

Hou,

; Zhang,

Saliency detection: A spectral residual approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1-8, 2007.

Crossref

[25]

Yan,

; Xu,

; Shi,

; Jia,

Hierarchical saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1155-1162, 2013.

Crossref

[26]

Yang,

; Zhang,

; Lu,

; Ruan,

; Yang,

M.-H.

Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3166-3173, 2013.

Crossref

[27]

Li,

; Yu,

Deep contrast learning for salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 478-487, 2016.

Crossref

[28]

Zhang,

D. W.

; Meng,

D. Y.

; Han,

J. W.

Co-saliency detection via a self-paced multiple-instance learning framework. IEEE Transactions on Pattern Analysisand Machine Intelligence Vol. 39, No. 5, 865-878, 2017.

Crossref Google Scholar

[29]

Zhang,

; Wang,

; Lu,

; Wang,

; Ruan,

Amulet: Aggregating multi-level convolutional features for salient object detection. In: Proceedings of the IEEE International Conference on Computer Vision, 202-211, 2017.

Crossref

[30]

Zhang,

; Wang,

; Lu,

; Wang,

; Yin,

Learning uncertain convolutional features for accurate saliency detection. In: Proceedings of the IEEE International Conference on Computer Vision, 212-221, 2017.

Crossref

[31]

Wang,

; Borji,

; Zhang,

; Lu,

A stagewise refinement model for detecting salient objects in images. In: Proceedings of the IEEE International Conference on Computer Vision, 4019-4028, 2017.

Crossref

[32]

Li,

; Yang,

; Cheng,

; Liu,

; Shen,

D. G.

Contour knowledge transfer for salient object detection. In: Computer Vision - ECCV 2018. Lecture Notes in Computer Science, Vol. 11219. Ferrari,

; Hebert,

; Sminchisescu,

; Weiss,

Eds. Springer Cham, 370-385, 2018.

[33]

Wang,

; Zhao,

; Shen,

; Hoi,

S. C.

; Borji,

Salient object detection with pyramid attention and salient edges. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1448-1457, 2019.

Crossref

[34]

Su,

; Li,

; Zhang,

; Xia,

; Tian,

Selectivity or invariance: Boundary-aware salient object detection. In: Proceedings of the IEEE International Conference on Computer Vision, 3799-3808, 2019.

Crossref

[35]

Zhao,

; Wu,

Pyramid feature attention network for saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3085-3094, 2019.

Crossref

[36]

Cong,

R. M.

; Lei,

J. J.

; Zhang,

C. Q.

; Huang,

Q. M.

; Cao,

X. C.

; Hou,

C. P.

Saliency detection for stereoscopic images based on depth confidence analysis and multiple cues fusion. IEEE Signal Processing Letters Vol. 23, No. 6, 819-823, 2016.

Crossref Google Scholar

[37]

Guo,

; Ren,

; Bei,

Salient object detection for RGB-D image via saliency evolution. In: Proceedings of the IEEE International Conference on Multimedia and Expo, 1-6, 2016.

Crossref

[38]

Fan,

D. P.

; Lin,

; Zhang,

; Zhu,

M. L.

; Cheng,

M. M.

Rethinking RGB-D salient object detection: Models, data sets, and large-scale benchmarks. IEEE Transactions on Neural Networks and Learning Systems , 2020.

Crossref Google Scholar

[39]

Zhang,

; Ren,

; Piao,

; Rong,

; Lu,

Select, supplement and focus for RGB-D saliency detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3469-3478, 2020.

Crossref

[40]

Piao,

; Rong,

; Zhang,

; Ren,

; Lu,

A2dele: Adaptive and attentive depth distiller for efficient RGB-D salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 9057-9066, 2020.

Crossref

[41]

Liu,

; Zhang,

; Han,

Learning selective self-mutual attention for RGB-D saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.

Crossref

[42]

Li,

G. Y.

; Liu,

; Ling,

H. B.

ICNet: Information conversion network for RGB-D based salient object detection. IEEE Transactions on Image Processing Vol. 29, 4873-4884, 2020.

Crossref Google Scholar

[43]

Fu,

; Fan,

D.-P.

; Ji,

G.-P.

; Zhao,

JL-DCF: Joint learning and densely-cooperative fusion framework for RGB-D salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3049-3059, 2020.

Crossref

[44]

Zhang,

; Fan,

D.-P.

; Dai,

; Anwar,

; Saleh,

F. S.

; Zhang,

; Barnes,

UC-Net: Uncertainty inspired RGB-D saliency detection via conditional variational autoencoders. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.

Crossref

[45]

Chen,

; Li,

Y. F.

CNN-based RGB-D salient object detection: Learn, select and fuse. arXiv preprint arXiv:1909.09309, 2019.

Google Scholar

[46]

Lang,

C. Y.

; Nguyen,

T. V.

; Katti,

; Yadati,

; Kankanhalli,

; Yan,

S. C.

Depth matters: influence of depth cues on visual saliency. In: Computer Vision - ECCV 2012. Lecture Notes in Computer Science, Vol. 7573. Fitzgibbon,

; Lazebnik,

; Perona,

; Sato,

; Schmid,

Eds. Springer Berlin Heidelberg, 101-115, 2012.

Crossref

[47]

Ciptadi,

; Hermans,

; Rehg,

J. M.

An in depth view of saliency. In: Proceedings of the 24th British Machine Vision Conference, 2013.

Crossref

[48]

Desingh,

; Madhava Krishna,

; Rajan,

; Jawahar,

C. V.

Depth really matters: Improving visual salient region detection with depth. In: Proceedings of the British Machine Vision Conference, 98.1-98.11, 2013.

Crossref

[49]

Cheng,

Y. P.

; Fu,

H. Z.

; Wei,

X. X.

; Xiao,

J. J.

; Cao,

X. C.

Depth enhanced saliency detection method. In: Proceedings of the International Conference on Internet Multimedia Computing and Service, 23-27, 2014.

Crossref

[50]

Ren,

; Gong,

; Yu,

; Zhou,

; Yang,

M. Y.

Exploiting global priors for RGB-D saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 25-32, 2015.

Crossref

[51]

Peng,

H. W.

; Li,

; Xiong,

W. H.

; Hu,

W. M.

; Ji,

R. R.

RGBD salient object detection: A benchmark and algorithms. In: Computer Vision - ECCV 2014. Lecture Notes in Computer Science, Vol. 8691. Fleet,

; Pajdla,

; Schiele,

; Tuytelaars,

Eds. Springer Cham, 92-109, 2014.

[52]

Qu,

L. Q.

; He,

S. F.

; Zhang,

J. W.

; Tian,

J. D.

; Tang,

Y. D.

; Yang,

Q. X.

RGBD salient object detection via deep fusion. IEEE Transactions on Image Processing Vol. 26, No. 5, 2274-2285, 2017.

Crossref Google Scholar

[53]

Zhao,

J.-X.

; Cao,

; Fan,

D.-P.

; Cheng,

M.-M.

; Li,

X.-Y.

; Zhang,

Contrast prior and fluid pyramid integration for RGBD salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3927-3936, 2019.

Crossref

[54]

Piao,

; Ji,

; Li,

; Zhang,

; Lu,

Depth-induced multi-scale recurrent attention network for saliency detection. In: Proceedings of the IEEE International Conference on Computer Vision, 7254-7263, 2019.

Crossref

[55]

Chen,

; Li,

Y. F.

; Su,

Multi-modal fusion network with multi-scale multi-path and cross-modal interactions for RGB-D salient object detection. Pattern Recognition Vol. 86, 376-385, 2019.

Crossref Google Scholar

[56]

Ju,

; Ge,

; Geng,

; Ren,

; Wu,

Depth saliency based on anisotropic center-surround difierence. In: Proceedings of the IEEE International Conference on Image Processing, 1115-1119, 2014.

Crossref

[57]

Feng,

; Barnes,

; You,

; McCarthy,

Local background enclosure for RGB-D salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2343-2350, 2016.

Crossref

[58]

Han,

J. W.

; Chen,

; Liu,

; Yan,

C. G.

; Li,

X. L.

CNNs-based RGB-D saliency detection via cross-view transfer and multiview fusion. IEEE Transactions on Cybernetics Vol. 48, No. 11, 3171-3183, 2018.

Crossref Google Scholar

[59]

Borji,

; Cheng,

M. M.

; Jiang,

H. Z.

; Li,

Salient object detection: A benchmark. IEEE Transactions on Image Processing Vol. 24, No. 12, 5706-5722, 2015.

Crossref Google Scholar

[60]

Cong,

; Lei,

; Fu,

; Cheng,

M.-M.

; Lin,

; Huang,

Review of visual saliency detection with comprehensive information. IEEE Transactions on Circuits and Systems for Video Technology Vol. 29, No. 10, 2941-2959, 2018.

Crossref Google Scholar

[61]

Zhang,

; Fu,

; Han,

; Borji,

; Li,

A review of co-saliency detection algorithms: Fundamentals, applications, and challenges. ACM Transactions on Intelligent Systems and Technology Vol. 9, No. 4, 1-31, 2018.

Crossref Google Scholar

[62]

Han,

J. W.

; Zhang,

D. W.

; Cheng,

; Liu,

; Xu,

Advanced deep-learning techniques for salient and category-specific object detection: A survey. IEEE Signal Processing Magazine Vol. 35, No. 1, 84-100, 2018.

Crossref Google Scholar

[63]

Nguyen,

T. V.

; Zhao,

; Yan,

S. C.

Attentive systems: A survey. International Journal of Computer Vision Vol. 126, No. 1, 86-110, 2018.

Crossref Google Scholar

[64]

Borji,

; Cheng,

M. M.

; Hou,

Q. B.

; Jiang,

H. Z.

; Li,

Salient object detection: A survey. Computational Visual Media Vol. 5, No. 2, 117-150, 2019.

Crossref Google Scholar

[65]

Zhao,

Z. Q.

; Zheng,

; Xu,

S. T.

; Wu,

X. D.

Object detection with deep learning: A review. IEEE Transactions on Neural Networks and Learning Systems Vol. 30, No. 11, 3212-3232, 2019.

Crossref Google Scholar

[66]

Wang,

W. G.

; Lai,

Q. X.

; Fu,

H. Z.

; Shen,

J. B.

; Ling,

H. B.

; Yang,

R. G.

Salient object detection in the deep learning era: An in-depth survey. arXiv preprint arXiv:1904.09146, 2019.

Google Scholar

[67]

Zhang,

; Lei,

; Fan,

; Wu,

; Zhang,

; Bu,

Depth combined saliency detection based on region contrast model. In: Proceedings of International Conference on Computer Science & Education, 763-766, 2012.

Crossref

[68]

Lei,

J. J.

; Zhang,

H. L.

; You,

; Hou,

C. P.

; Wang,

L. H.

Evaluation and modeling of depth feature incorporated visual attention for salient object segmentation. Neurocomputing Vol. 120, 24-33, 2013.

Crossref Google Scholar

[69]

Fan,

; Liu,

; Sun,

Salient region detection for stereoscopic images. In: Proceedings of the International Conference on Digital Signal Processing, 454-458, 2014.

Crossref

[70]

Guo,

J. F.

; Ren,

T. W.

; Bei,

; Zhu,

Y. J.

Salient object detection in RGB-D image based on saliency fusion and propagation. In: Proceedings of the 7th International Conference on Internet Multimedia Computing and Service, Article No. 59, 2015.

Crossref

[71]

Tang,

Y. L.

; Tong,

R. F.

; Tang,

; Zhang,

Depth incorporating with color improves salient object detection. The Visual Computer Vol. 32, No. 1, 111-121, 2016.

Crossref Google Scholar

[72]

Jiang,

; Koch,

; Zell,

Salient regions detection for indoor robots using RGB-D data. In: Proceedings of the IEEE International Conference on Robotics and Automation, 1323-1328, 2015.

Crossref

[73]

Xue,

; Gu,

; Li,

; Yang,

RGB-D saliency detection via mutual guided manifold ranking. In: Proceedings of IEEE International Conference on Image Processing, 666-670, 2015.

Crossref

[74]

Zhu,

; Cao,

; Fang,

; Xiao,

; Wu,

; Deng,

; Liu,

Selective features for RGB-D saliency. In: Proceedings of Chinese Automation Congress, 512-517, 2015.

Crossref

[75]

Du,

; Liu,

; Song,

H. K.

; Mei,

; Xu,

Improving RGBD saliency detection using progressive region classification and saliency fusion. IEEE Access Vol. 4, 8987-8994, 2016.

Crossref Google Scholar

[76]

Wang,

S.-T.

; Zhou,

; Qu,

H.-B.

; Li,

RGBD saliency detection under bayesian framework. In: Proceedings of the 23rd International Conference on Pattern Recognition, 1881-1886, 2016.

[77]

Sheng,

; Liu,

; Zhang,

Saliency analysis based on depth contrast increased. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 1347-1351, 2016.

Crossref

[78]

Song,

; Liu,

; Du,

; Sun,

Depth-aware saliency detection using discriminative saliency fusion. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 1626-1630, 2016.

Crossref

[79]

Wang,

S. T.

; Zhou,

; Qu,

H. B.

; Li,

Visual saliency detection for RGB-D images with generative model. In: Computer Vision - ACCV 2016. Lecture Notes in Computer Science, Vol. 10115. Lai,

S. H.

; Lepetit,

; Nishino,

; Sato,

Eds. Springer Cham, 20-35, 2017.

[80]

Feng,

; Barnes,

; You,

HOSO: Histogram of surface orientation for RGB-D salient object detection. In Proceedings of the International Conference on Digital Image Computing: Techniques and Applications, 1-8, 2017.

Crossref

[81]

Chen,

; Li,

Y.-F.

; Su,

^{3}

Net: Multi-scale multi-path multi-modal fusion network and example application to RGB-D salient object detection. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 4911-4916, 2017.

Crossref

[82]

Chen,

; Li,

Y. F.

; Su,

RGB-D saliency detection by multi-stream late fusion network. In: Computer Vision Systems. Lecture Notes in Computer Science, Vol. 10528. Liu,

; Chen,

; Vincze,

Eds. Springer Cham, 459-468, 2017.

Crossref

[83]

Shigematsu,

; Feng,

; You,

; Barnes,

Learning RGB-D salient object detection using background enclosure, depth contrast, and top-down features. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2749-2757, 2017.

Crossref

[84]

Zhu,

; Li,

; Wang,

An innovative salient object detection using center-dark channel prior. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 1509-1515, 2017.

Crossref

[85]

Zhu,

; Li,

A three-pathway psychobiological framework of salient object detection using stereoscopic technology. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 3008-3014, 2017.

[86]

Wang,

A. Z.

; Wang,

M. H.

RGB-D salient object detection via minimum barrier distance transform and saliency fusion. IEEE Signal Processing Letters Vol. 24, No. 5, 663-667, 2017.

Crossref Google Scholar

[87]

Song,

H. K.

; Liu,

; Du,

; Sun,

G. L.

; Le Meur,

, Ren,

T. W.

Depth-aware salient object detection and segmentation via multiscale discriminative saliency fusion and bootstrap learning. IEEE Transactions on Image Processing Vol. 26, No. 9, 4204-4216, 2017.

Crossref Google Scholar

[88]

Cong,

R. M.

; Lei,

J. J.

; Fu,

H. Z.

; Lin,

W. S.

; Huang,

Q. M.

; Cao,

X. C.

; Hou,

C. P.

An iterative co-saliency framework for RGBD images. IEEE Transactions on Cybernetics Vol. 49, No. 1, 233-246, 2019.

Crossref Google Scholar

[89]

Imamoglu,

; Shimoda,

; Zhang,

; Fang,

Y. M.

; Kanezaki,

; Yanai,

; Nishida,

An integration of bottom-up and top-down salient cues on RGB-D data: Saliency from objectness versus non-objectness. Signal, Image and Video Processing Vol. 12, No. 2, 307-314, 2018.

Crossref Google Scholar

[90]

Cong,

R. M.

; Lei,

J. J.

; Fu,

H. Z.

; Huang,

Q. M.

; Cao,

X. C.

; Ling,

HSCS: Hierarchical sparsity based Co-saliency detection for RGBD images. IEEE Transactions on Multimedia Vol. 21, No. 7, 1660-1671, 2019.

Crossref Google Scholar

[91]

Cong,

R. M.

; Lei,

J. J.

; Fu,

H. Z.

; Huang,

Q. M.

; Cao,

X. C.

; Hou,

C. P.

Co-saliency detection for RGBD images based on multi-constraint feature matching and cross label propagation. IEEE Transactions on Image Processing Vol. 27, No. 2, 568-579, 2018.

Crossref Google Scholar

[92]

Chen,

; Li,

Progressively complementarity-aware fusion network for RGB-D salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3051-3060, 2018.

Crossref

[93]

Huang,

; Shen,

C.-H.

; Hsiao,

H.-F.

RGBD salient object detection using spatially coherent deep learning framework. In: Proceedings of the IEEE International Conference on Digital Signal Processing, 1-5, 2018.

Crossref

[94]

Chen,

; Li,

Y.-F.

; Su,

Attention-aware crossmodal cross-level fusion network for RGB-D salient object detection. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 6821-6826, 2018.

Crossref

[95]

Liang,

F. F.

; Duan,

L. J.

; Ma,

; Qiao,

Y. H.

; Cai,

; Qing,

L. Y.

Stereoscopic saliency model using contrast and depth-guided-background prior. Neurocomputing Vol. 275, 2227-2238, 2018.

Crossref Google Scholar

[96]

Liu,

Z. Y.

; Shi,

; Duan,

Q. T.

; Zhang,

; Zhao,

Salient object detection for RGB-D image by single stream recurrent convolution neural network. Neurocomputing Vol. 363, 46-57, 2019.

Crossref Google Scholar

[97]

Huang,

; Xing,

; Wang,

Z. Z.

RGB-D salient object detection by a CNN with multiple layers fusion. IEEE Signal Processing Letters Vol. 26, No. 4, 552-556, 2019.

Crossref Google Scholar

[98]

Liu,

; Hu,

; Zhang,

; Chen,

Two-stream refinement network for RGB-D saliency detection. In: Proceedings of IEEE International Conference on Image Processing, 3925-3929, 2019.

Crossref

[99]

Du,

; Liu,

; Shi,

Salient object segmentation based on depth-aware image layering. Multimedia Tools and Applications Vol. 78, No. 9, 12125-12138, 2019.

Crossref Google Scholar

[100]

Zhou,

W. J.

; Lv,

, Lei,

J. S.

; Yu,

Global and local-contrast guides content-aware fusion for RGB-D saliency prediction. IEEE Transactions on Systems, Man, and Cybernetics: Systems , 2019.

Crossref Google Scholar

[101]

Ma,

C. Y.

; Hang,

H. M.

Learning-based saliency model with depth information. Journal of Vision Vol. 15, No. 6, 19, 2015.

Crossref Google Scholar

[102]

Zhu,

; Cai,

; Huang,

; Li,

T. H.

; Li,

PDNet: Prior-model guided depth-enhanced network for salient object detection. In: Proceedings of the IEEE International Conference on Multimedia and Expo, 199-204, 2019.

Crossref

[103]

Chen,

; Li,

Y. F.

Three-stream attention-aware network for RGB-D salient object detection. IEEE Transactions on Image Processing Vol. 28, No. 6, 2825-2835, 2019.

Crossref Google Scholar

[104]

Chen,

; Li,

Y. F.

; Su,

Discriminative cross-modal transfer learning and densely cross-level feedback fusion for RGB-D salient object detection. IEEE Transactions on Cybernetics Vol. 50, No. 11, 4808-4820, 2020.

Crossref Google Scholar

[105]

Cong,

R. M.

; Lei,

J. J.

; Fu,

H. Z.

; Hou,

J. H.

; Huang,

Q. M.

; Kwong,

Going from RGB to RGBD saliency: A depth-guided transformation model. IEEE Transactions on Cybernetics Vol. 50, No. 8, 3627-3639, 2020.

Crossref Google Scholar

[106]

Wang,

N. N.

; Gong,

X. J.

Adaptive fusion for RGB-D salient object detection. IEEE Access Vol. 7, 55277-55284, 2019.

Crossref Google Scholar

[107]

Jin,

Z. G.

; Li,

J. K.

; Li,

Co-saliency detection for RGBD images based on effective propagation mechanism. IEEE Access Vol. 7, 141311-141318,2019.

Crossref Google Scholar

[108]

Ding,

; Liu,

; Huang,

M. K.

; Shi,

; Wang,

X. Y.

Depth-aware saliency detection using convolutional neural networks. Journal of Visual Communication and Image Representation Vol. 61, 1-9, 2019.

Crossref Google Scholar

[109]

Chen,

; Huang,

Depth potentiality-aware gated attention network for RGB-D salient object detection. arXiv preprint arXiv:2003.08608, 2020.

Google Scholar

[110]

Wang,

; Li,

Y. K.

; Elder,

J. H.

; Lu,

H. C.

; Wu,

R. M.

; Zhang,

Synergistic saliency and depth prediction for RGB-D saliency detection. arXiv preprint arXiv:2007.01711, 2020.

Google Scholar

[111]

Zhou,

X. F.

; Li,

G. Y.

; Gong,

; Liu,

; Zhang,

J. Y.

Attention-guided RGBD saliency detection using appearance information. Image and Vision Computing Vol. 95, 103888, 2020.

Crossref Google Scholar

[112]

Liu,

Z. Y.

; Zhang,

; Zhao,

A cross-modal adaptive gated fusion generative adversarial network for RGB-D salient object detection. Neurocomputing Vol. 387, 210-220, 2020.

Crossref Google Scholar

[113]

Liang,

F. F.

; Duan,

L. J.

; Ma,

; Qiao,

Y. H.

; Cai,

; Miao,

; Ye,

CoCNN: RGB-D deep fusion for stereoscopic salient object detection. Pattern Recognition Vol. 104, 107329, 2020.

Crossref Google Scholar

[114]

Jiang,

; Zhou,

Z. T.

; Wang,

; Tang,

; Luo,

cmSalGAN: RGB-D salient object detection with cross-view generative adversarial networks. IEEE Transactions on Multimedia , 2020.

Crossref Google Scholar

[115]

Xiao,

; Li,

; Peng,

Y. M.

; Cao,

C. H.

; Hu,

; Gao,

X. P.

Multi-modal weights sharing and hierarchical feature fusion for RGBD salient object detection. IEEE Access Vol. 8, 26602-26611, 2020.

Crossref Google Scholar

[116]

Zhang,

; Lin,

; Xu,

; Jin,

W. D.

; Lu,

S. P.

; Fan,

D. P.

Bilateral attention network for RGB-D salient object detection. arXiv preprint arXiv:2004.14582, 2020.

Google Scholar

[117]

Li,

C. Y.

; Cong,

R. M.

; Kwong,

; Hou,

J. H.

; Fu,

H. Z.

; Zhu,

G. P.

; Zhang,

; Huang,

ASIF-Net: Attention steered interweave fusion network for RGB-D salient object detection. IEEE Transactions on Cybernetics , 2020.

Crossref Google Scholar

[118]

Huang,

; Xing,

; Zou,

Y. B.

Triple-complementary network for RGB-D salient object detection. IEEE Signal Processing Letters Vol. 27, 775-779, 2020.

Crossref Google Scholar

[119]

Chen,

; Wei,

J. P.

; Peng,

; Zhang,

W. Z.

; Qin,

Improved saliency detection in RGB-D images using two-phase depth estimation and selective deep fusion. IEEE Transactions on Image Processing Vol. 29, 4296-4307, 2020.

Crossref Google Scholar

[120]

Zhou,

W. J.

; Chen,

Y. Z.

; Liu,

; Yu,

GFNet: Gate fusion network with Res2Net for detecting salient objects in RGB-D images. IEEE Signal Processing Letters Vol. 27, 800-804, 2020.

Crossref Google Scholar

[121]

Liu,

Z. Y.

; Tang,

J. T.

; Xiang,

; Zhao,

Salient object detection for RGB-D images by generative adversarial network. Multimedia Tools and Applications Vol. 79, Nos. 35-36, 25403-25425, 2020.

Crossref Google Scholar

[122]

Li,

G. Y.

; Liu,

; Ye,

L. W.

; Wang,

; Ling,

H. B.

Cross-modal weighting network for RGB-D salient object detection. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12362. Vedaldi,

; Bischof,

; Brox,

; Frahm,

J. M.

Eds. Springer Cham, 665-681, 2020.

[123]

Pang,

Y. W.

; Zhang,

L. H.

; Zhao,

X. Q.

; Lu,

H. C.

Hierarchical dynamic filtering network for RGB-D salient object detection. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12370. Vedaldi,

; Bischof,

; Brox,

; Frahm,

J. M.

Eds. Springer Cham, 235-252, 2020.

[124]

Luo,

; Li,

; Yang,

; Jiao,

Z. C.

; Cheng,

; Lyu,

S. W.

Cascade graph neural networks for RGB-D salient object detection. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12357. Vedaldi,

; Bischof,

; Brox,

; Frahm,

J. M.

Eds. Springer Cham, 346-364, 2020.

[125]

Li,

C. Y.

; Cong,

R. M.

; Piao,

Y. R.

; Xu,

Q. Q.

; Loy,

C. C.

RGB-D salient object detection with cross-modality modulation and selection. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12353. Vedaldi,

; Bischof,

; Brox,

; Frahm,

J. M.

Eds. Springer Cham, 225-241, 2020.

[126]

Zhao,

X. Q.

; Zhang,

L. H.

; Pang,

Y. W.

; Lu,

H. C.

; Zhang,

A single stream network for robust and real-time RGB-D salient object detection. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12367. Vedaldi,

; Bischof,

; Brox,

; Frahm,

J. M.

Eds. Springer Cham, 646-662, 2020.

[127]

Ji,

; Li,

J. J.

; Zhang,

; Piao,

Y. R.

; Lu,

H. C.

Accurate RGB-D salient object detection via collaborative learning. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12363. Vedaldi,

; Bischof,

; Brox,

; Frahm,

J. M.

Eds. Springer Cham, 52-69, 2020.

[128]

Fan,

D. P.

; Zhai,

Y. J.

; Borji,

; Yang,

J. F.

; Shao,

BBS-Net: RGB-D salient object detection with a bifurcated backbone strategy network. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12357. Vedaldi,

; Bischof,

; Brox,

; Frahm,

J. M.

Eds. Springer Cham, 275-292,2020.

[129]

Zhang,

; Fei,

S. X.

; Liu,

; Xu,

; Piao,

Y. R.

; Lu,

H. C.

Asymmetric two-stream architecture for accurate RGB-D saliency detection. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12373. Vedaldi,

; Bischof,

; Brox,

; Frahm,

J. M.

Eds. Springer Cham, 374-390, 2020.

[130]

Chen,

S. H.

; Fu,

Progressively guided alternate refinement network for RGB-D salient object detection. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12353. Vedaldi,

; Bischof,

; Brox,

; Frahm,

J. M.

Eds. Springer Cham, 520-538, 2020.

[131]

Huang,

; Chen,

H. X.

; Zhou,

; Yang,

Y. Z.

; Wang,

C. Y.

Multi-level cross-modal interaction network for RGB-D salient object detection. arXiv preprint arXiv:2007.14352, 2020.

Google Scholar

[132]

Wang,

X. H.

; Li,

; Chen,

, Fang,

Y. M.

; Hao,

A. M.

; Qin,

Data-level recombination and lightweight fusion scheme for RGB-D salient object detection. IEEE Transactions on Image Processing Vol. 30, 458-471, 2021.

Crossref Google Scholar

[133]

Wang,

; Li,

; Chen,

; Hao,

; Qin,

Knowing depth quality in advance: A depth quality assessment method for RGB-D salient object detection. arXiv preprint arXiv:2008.04157, 2020.

Google Scholar

[134]

Chen,

; Wei,

; Peng,

; Qin,

Depth quality aware salient object detection. arXiv preprint arXiv:2008.04159, 2020.

Google Scholar

[135]

Zhao,

J. W.

; Zhao,

Y. F.

; Li,

; Chen,

X. W.

Is depth really necessary for salient object detection. arXiv preprint arXiv:2006.00269, 2020.

Google Scholar

[136]

Chen,

; Deng,

Y. J.

; Li,

Y. F.

; Hung,

T. Y.

; Lin,

G. S.

RGBD salient object detection via disentangled cross-modal fusion. IEEE Transactions on Image Processing Vol. 29, 8407-8416, 2020.

Crossref Google Scholar

[137]

Piao,

Y. R.

; Li,

; Zhang,

; Yu,

J. Y.

; Lu,

H. C.

Saliency detection via depth-induced cellular automata on light field. IEEE Transactions on Image Processing Vol. 29, 1879-1889, 2020.

Crossref Google Scholar

[138]

Zhang,

; Zhang,

; Piao,

Y. R.

; Hu,

B. Q.

; Lu,

H. C.

Feature reintegration over differential treatment: A top-down and adaptive fusion network for RGB-D salient object detection. In: Proceedings of the 28th ACM International Conference on Multimedia, 4107-4115, 2020.

Crossref

[139]

Niu,

; Geng,

; Li,

; Liu,

Leveraging stereopsis for saliency analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 454-461, 2012.

[140]

Li,

; Ye,

; Ji,

; Ling,

; Yu,

Saliency detection on light field. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2806-2813, 2014.

Crossref

[141]

Zhang,

; Ji,

; Piao,

Y. R.

; Li,

J. J.

; Zhang,

; Xu,

; Lu,

LFNet: Light field fusion network for salient object detection. IEEE Transactions on Image Processing Vol. 29, 6276-6287, 2020.

Crossref Google Scholar

[142]

Li,

; Sun,

; Yu,

A weighted sparse coding framework for saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5216-5223, 2015.

Crossref

[143]

Zhang,

; Wang,

; Gao,

; Wang,

; Zhang,

; Wu,

Saliency detection with a deeper investigation of light field. In: Proceedings of the International Joint Conference on Artificial Intelligence, 2212-2218, 2015.

[144]

Sheng,

; Zhang,

; Liu,

; Xiong,

Relative location for light field saliency detection. In: Proceedings of the IEEE Conference on Acoustics, Speech and Signal Processing, 1631-1635, 2016.

Crossref

[145]

Zhang,

; Wang,

; Lin,

; Yang,

; Gao,

; Rui,

Saliency detection on light field. ACM Transactions on Multimedia Computing, Communications, and Applications Vol. 13, No. 3, 1-22, 2017.

Crossref Google Scholar

[146]

Wang,

A. Z.

; Wang,

M. H.

; Li,

X. Y.

; Mi,

Z. T.

; Zhou,

A two-stage Bayesian integration framework for salient object detection on light field. Neural Processing Letters Vol. 46, No. 3, 1083-1094, 2017.

Crossref Google Scholar

[147]

Li,

N. Y.

; Ye,

J. W.

; Ji,

; Ling,

H. B.

; Yu,

J. Y.

Saliency detection on light field. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 8, 1605-1616, 2017.

Crossref Google Scholar

[148]

Li,

; Zhan,

; Zhang,

; Sheng,

Saliency detection with relative location measure in light field image. In: Proceedings of the International Conference on Image, Vision and Computing, 8-12, 2017.

[149]

Wang,

; Liao,

; Surman,

; Tu,

; Zheng,

; Yuan,

Salience guided depth calibration for perceptually optimized compressive light field 3D display. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2031-2040, 2018.

Crossref

[150]

Piao,

Y. R.

; Li,

; Zhang,

Depth-induced cellular automata for light field saliency. In: Proceedings of the Frontiers in Optics/Laser Science, OSA Technical Digest, FTh3E.3, 2018.

Crossref

[151]

Wang,

; Piao,

; Li,

; Zhang,

; Lu,

Deep learning for light field saliency detection. In: Proceedings of the IEEE International Conference on Computer Vision, 8838-8848, 2019.

Crossref

[152]

Piao,

Y. R.

; Rong,

Z. K.

; Zhang,

; Li,

; Lu,

H. C.

Deep light-field-driven saliency detection from a single view. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, 904-911, 2019.

Crossref

[153]

Zhang,

; Li,

; WEI,

; Piao,

; Lu,

Memory-oriented decoder for light field salient object detection. In: Proceedings of the International Conference on Neural Information Processing Systems, 896-906, 2019.

[154]

Piao,

Y. R.

; Rong,

Z. K.

; Zhang,

; Lu,

H. C.

Exploit and replace: An asymmetrical two-stream architecture for versatile light field saliency detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, No. 7, 11865-11873, 2020.

Crossref

[155]

Wang,

; Dong,

Y. Y.

; Zhang,

; Wang,

Regionbased depth feature descriptor for saliency detection on light field. Multimedia Tools and Applications , 2020.

Crossref Google Scholar

[156]

Zhang,

; Liu,

Y. M.

; Zhang,

S. P.

; Poppe,

, Wang,

Light field saliency detection with deep convolutional networks. IEEE Transactions on Image Processing Vol. 29, 4421-4434, 2020.

Crossref Google Scholar

[157]

Achanta,

; Hemami,

; Estrada,

; Susstrunk,

Frequency-tuned salient region detection. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 1597-1604, 2009.

Crossref

[158]

Krahenbuhl,

Saliency filters: Contrast based filtering for salient region detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 733-740, 2012.

[159]

Fan,

D.-P.

; Cheng,

M.-M.

; Liu,

; Li,

; Borji.

Structure-measure: A new way to evaluate foreground maps. In: Proceedings of the IEEE International Conference on Computer Vision, 4548-4557, 2017.

Crossref

[160]

Fan,

D. P.

; Gong,

; Cao,

; Ren,

; Cheng,

M. M.

; Borji,

Enhanced-alignment measure for binary foreground map evaluation. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, 698-704, 2018.

Crossref

[161]

Qin,

; Lu,

; Xu,

; Wang,

Saliency detection via cellular automata. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 110-119, 2015.

Crossref

[162]

Zhou,

; Yang,

Z. H.

; Yuan,

; Zhou,

Z. T.

; Hu,

D. W.

Salient region detection via integrating diffusion-based compactness and local contrast. IEEE Transactions on Image Processing Vol. 24, No. 11, 3308-3320, 2015.

Crossref Google Scholar

[163]

Huang,

X. M.

; Zhang,

Y. J.

300-FPS salient object detection via minimum directional contrast. IEEE Transactions on Image Processing Vol. 26, No. 9, 4243-4254, 2017.

Crossref Google Scholar

[164]

Huang,

; Qi,

J. Q.

; Lu,

H. C.

; Zhang,

L. H.

; Ruan,

Salient object detection via multiple instance learning. IEEE Transactions on Image Processing Vol. 26, No. 4, 1911-1922, 2017.

Crossref Google Scholar

[165]

Huang,

X. M.

; Zhang,

Y. J.

Water flow driven salient object detection at 180 fps. Pattern Recognition Vol. 76, 95-107, 2018.

Crossref Google Scholar

[166]

Xu,

; Tao,

D. C.

; Xu,

Multi-view learning with incomplete views. IEEE Transactions on Image Processing Vol. 24, No. 12, 5812-5825, 2015.

Crossref Google Scholar

[167]

Zhou,

; Thung,

K. H.

; Zhu,

X. F.

; Shen,

D. G.

Effective feature learning and fusion of multimodality data using stage-wise deep neural network for dementia diagnosis. Human Brain Mapping Vol. 40, No. 3, 1001-1016, 2019.

Crossref Google Scholar

[168]

Zhou,

; Liu,

M. X.

; Thung,

K. H.

; Shen,

D. G.

Latent representation learning for Alzheimer’s disease diagnosis with incomplete multi-modality neuroimaging and genetic data. IEEE Transactions on Medical Imaging Vol. 38, No. 10, 2411-2422, 2019.

Crossref Google Scholar

[169]

Zhou,

; Thung,

K. H.

; Liu,

M. X.

; Shi,

; Zhang,

C. Q.

; Shen,

D. G.

Multi-modal latent space inducing ensemble SVM classifier for early dementia diagnosis with neuroimaging data. Medical Image Analysis Vol. 60, 101630, 2020.

Crossref Google Scholar

[170]

Zhou,

; Fu,

H. Z.

; Chen,

; Shen,

J. B.

; Shao,

Hi-net: Hybrid-fusion network for multi-modal MR image synthesis. IEEE Transactions on Medical Imaging Vol. 39, No. 9, 2772-2781, 2020.

Crossref Google Scholar

[171]

Godard,

; Aodha,

O. M.

; Brostow,

G. J.

Unsupervised monocular depth estimation with left-right consistency. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6602-6611, 2017.

Crossref

[172]

Liu,

; Shen,

; Lin,

Deep convolutional neural fields for depth estimation from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5162-5170, 2015.

Crossref

[173]

Wang,

; Zhang,

; Wang,

; Lin,

; Lu,

SDC-depth: Semantic divide-and-conquer network for monocular depth estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 541-550, 2020.

Crossref

[174]

Jin,

; Xu,

; Zheng,

; Zhang,

; Tang,

; Xu,

; Yu,

; Gao,

Geometric structure based and regularized depth estimation from 360 indoor imagery. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 889-898, 2020.

Crossref

[175]

Mirza,

; Osindero,

Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.

Google Scholar

[176]

Zhu,

D. D.

; Dai,

; Luo,

; Zhang,

G. K.

; Lu,

J. W.

Multi-scale adversarial feature learning for saliency detection. Symmetry Vol. 10, No. 10, 457, 2018.

Crossref Google Scholar

[177]

Pan,

J. T.

; Ferrer,

C. C.

; McGuinness,

; O’Connor,

N. E.

; Torres,

; Sayrol,

; Giro-i-Nieto,

SalGAN: Visual saliency prediction with generative adversarial networks. arXiv preprint arXiv:1701.01081, 2017.

Google Scholar

[178]

Vaswani,

; Shazeer,

; Parmar,

; Uszkoreit,

; Jones,

; Gomez,

A. N.

; Kaiser,

; Polosukhin,

Attention is all you need. In: Proceedings of the Conference on Neural Information Processing Systems, 5998-6008, 2017.

[179]

Wang,

; Jiang,

; Qian,

; Yang,

; Li,

; Zhang,

; Wang,

; Tang,

Residual attention network for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3156-3164, 2017.

Crossref

[180]

Fang,

H. S.

; Cao,

J. K.

; Tai,

Y. W.

; Lu,

C. W.

Pairwise body-part attention for recognizing human-object interactions. In: Computer Vision - ECCV 2018. Lecture Notes in Computer Science, Vol. 11214. Ferrari,

; Hebert,

; Sminchisescu,

; Weiss,

Eds. Springer Cham, 52-68, 2018.

[181]

Wang,

W. G.

; Shen,

J. B.

Deep visual attention prediction. IEEE Transactions on Image Processing Vol. 27, No. 5, 2368-2378, 2018.

Crossref Google Scholar

[182]

Lu,

; Yang,

; Batra,

; Parikh,

Hierarchical question-image co-attention for visual question answering. In: Proceedings of the International Conference on Neural Information Processing Systems, 289-297, 2016.

[183]

Yu,

; Yu,

; Cui,

; Tao,

; Tian,

Deep modular co-attention networks for visual question answering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6281-6290, 2019.

Crossref

[184]

Lu,

; Wang,

; Ma,

; Shen,

; Shao,

; Porikli,

See more, know more: Unsupervised video object segmentation with co-attention siamese networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3623-3632, 2019.

Crossref

[185]

Zeng,

; Zhuge,

; Lu,

; Zhang,

; Qian,

; Yu,

Multi-source weak supervision for saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6074-6083, 2019.

Crossref

[186]

Zhang,

D. W.

; Meng,

D. Y.

; Zhao,

; Han,

J. W.

Bridging saliency detection to weakly supervised object detection based on self-paced curriculum learning. arXiv preprint arXiv:1703.01290, 2017.

Google Scholar

[187]

Qian,

M. Y.

; Qi,

J. Q.

; Zhang,

L. H.

; Feng,

M. Y.

; Lu,

H. C.

Language-aware weak supervision for salient object detection. Pattern Recognition Vol. 96, 106955, 2019.

Crossref Google Scholar

[188]

Yan,

; Li,

; Xie,

; Li,

; Wang,

; Chen,

; Lin,

Semi-supervised video salient object detection using pseudo-labels. In: Proceedings of the IEEE International Conference on Computer Vision, 7284-7293, 2019.

Crossref

[189]

Zhou,

; Huo,

S. W.

; Xiang,

; Hou,

C. P.

; Kung,

S. Y.

Semi-supervised salient object detection using a linear feedback control system model. IEEE Transactions on Cybernetics Vol. 49, No. 4, 1173-1185, 2019.

Crossref Google Scholar

[190]

Zhang,

; Han,

; Zhang,

Supervision by fusion: Towards unsupervised learning of deep salient object detector. In: Proceedings of the IEEE International Conference on Computer Vision, 4048-4056, 2017.

Crossref

[191]

Chen,

; Liu,

; Chang,

; Cheng,

; Amini,

; Wang,

Adversarial robustness: From self-supervised pre-training to fine-tuning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 699-708, 2020.

Crossref

[192]

Dai,

; Diller,

; Niefiner,

SG-NN: Sparse generative neural networks for self-supervised scene completion of RGB-D scans. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 849-858, 2020.

Crossref

[193]

Lai,

; Bo,

; Ren,

; Fox,

A largescale hierarchical multi-view RGB-D object dataset. In: Proceedings of the IEEE International Conference on Robotics and Automation, 1817-1824, 2011.

Crossref

[194]

Zhang,

; Li,

W. Q.

; Wang,

P. C.

; Ogunbona,

, Liu,

; Tang,

A large scale RGB-D dataset for action recognition. In: Understanding Human Activities Through 3D Sensors. Lecture Notes in Computer Science, Vol. 10188. Wannous,

; Pala,

; Daoudi,

; Flórez-Revuelta,

Eds. Springer Cham, 101-114, 2018.

[195]

He,

Y. H.

; Lin,

; Liu,

Z. J.

; Wang,

H. R.

; Li,

L. J.

; Han,

AMC: AutoML for model compression and acceleration on mobile devices. In: Computer Vision - ECCV 2018. Lecture Notes in Computer Science, Vol. 11211. Ferrari,

; Hebert,

; Sminchisescu,

; Weiss,

Eds. Springer Cham, 815-832, 2018.

[196]

Cheng,

; Wang,

; Zhou,

; Zhang,

A survey of model compression and acceleration for deep neural networks. arXiv preprint arXiv:1710.09282, 2017.

Google Scholar

[197]

Ma,

; Sun,

; Meng,

; Ding,

; Li,

Learning multiscale deep features and SVM regressors for adaptive RGB-T saliency detection. In: Proceedings of the 10th International Symposium on Computational Intelligence and Design, 389-392, 2017.

Crossref

[198]

Wang,

G. Z.

; Li,

C. L.

; Ma,

Y. P.

; Zheng,

A. H.

; Tang,

; Luo,

RGB-T saliency detection benchmark: Dataset, baselines, analysis and a novel approach. In: Image and Graphics Technologies and Applications. Communications in Computer and Information Science, Vol. 875. Wang,

; Jiang,

; Peng,

Eds. Springer Singapore, 359-369, 2018.

Crossref

[199]

Wang,

G. Z.

; Li,

C. L.

; Ma,

Y. P.

; Zheng,

A. H.

; Tang,

; Luo,

; Jiang,

; Peng,

Eds. Springer Singapore, 359-369, 2018.

Crossref

[200]

Sun,

D. D.

; Li,

; Ding,

Z. L.

; Luo,

RGB-T saliency detection via robust graph learning and collaborative manifold ranking. In: Bio-inspired Computing: Theories and Applications. Communications in Computer and Information Science, Vol. 1160. Pan,

; Liang,

; Qu,

Eds. Springer Singapore, 670-684, 2020.

Crossref

[201]

Tu,

; Xia,

; Li,

; Lu,

; Tang,

M3S-NIR: Multi-modal multi-scale noise-insensitive ranking for RGB-T saliency detection. In: Proceedings of the IEEE Conference on Multimedia Information Processing and Retrieval, 141-146, 2019.

Crossref

[202]

Tu,

Z. Z.

; Li,

C. L.

; Lang,

; Tang,

Multi-interactive encoder-decoder network for RGBT salient object detection. arXiv preprint arXiv:2005.02315, 2020.

Google Scholar

[203]

Tu,

Z. Z.

; Xia,

; Li,

C. L.

; Wang,

X. X.

; Ma,

; Tang,

RGB-T image saliency detection via collaborative graph learning. IEEE Transactions on Multimedia Vol. 22, No. 1, 160-173, 2020.

Crossref Google Scholar

[204]

Zhang,

; Huang,

N. C.

; Yao,

; Zhang,

D. W.

; Shan,

C. F.

; Han,

J. G.

RGB-T salient object detection via fusing multi-level CNN features. IEEE Transactions on Image Processing Vol. 29, 3321-3335, 2020.

Crossref Google Scholar

[205]

Tu,

Z. Z.

; Ma,

; Li,

C. L.

; Xu,

J. M.

; Liu,

Y. T.

RGBT salient object detection: A large-scale dataset and benchmark. arXiv preprint arXiv:2007.03262, 2020.

Google Scholar

Computational Visual Media

Volume 7 Issue 1,
March 2021

Pages 37-69

DOI: 10.1007/s41095-020-0199-z

Cite this article:

Zhou T, Fan D-P, Cheng M-M, et al. RGB-D salient object detection: A survey. Computational Visual Media, 2021, 7(1): 37-69. https://doi.org/10.1007/s41095-020-0199-z

Return

10.1007/s41095-020-0199-z.T001Table 1Summary of RGB-D based salient object detection methods published from 2012 to 2016

No.	Year	Method	Pub.	Training set	Backbone	Description
1	2012	DM [46]	ECCV	Without	Without	Models the correlation between saliency and depth by approximating the joint density using Gaussian mixture models
2	2012	RCM [67]	ICCSE	Without	Without	Develops a region contrast based salient object detection model with depth cues
3	2013	LS [47]	BMVC	Without	Without	Extends the dissimilarity framework to model the joint interaction between depth cues and RGB images
4	2013	RC [48]	BMVC	Without	Without	Derives RGB-D saliency by formulating a 3D saliency model based on the region contrast of the scene and fuses it using SVM
5	2013	SOS [68]	NEURO	Without	Without	Incorporates depth cues for salient object segmentation by suppressing background regions
6	2014	SRDS [69]	ICDSP	Without	Without	Integrates depth and depth weighted color contrast with spatial compactness of color distribution
7	2014	LHM [51]	ECCV	Without	Without	Uses a multi-stage RGB-D algorithm to combine both depth and appearance cues to segment salient objects
8	2014	DESM [49]	ICIMCS	Without	Without	Combines three saliency cues: color contrast, spatial bias, and depth contrast
9	2014	ACSD [56]	ICIP	Without	Without	Measures a point’s saliency by how much it stands out from the surroundings, and has two priors (regions nearer to viewers are more salient and salient objects tend to be located at the center)
10	2015	GP [50]	CVPRW	Without	Without	Explores orientation and background priors for detecting salient objects, and uses PageRank and MRFs to optimize the saliency maps
11	2015	SFP [70]	ICIMCS	Without	Without	Develops a RGB-D based salient object detection approach using saliency fusion and propagation
12	2015	DIC [71]	TVC	Without	Without	Fuses the saliency maps from color and depth to generate a noise-free salient patch, and utilizes random walk algorithm to infer the object boundary
13	2015	SRD [72]	ICRA	Without	Without	Designs a graph-based segmentation to identify homogeneous regions using color and depth cues
14	2015	MGMR [73]	ICIP	Without	Without	Designs a mutual guided manifold ranking strategy to achieve salient object detection
15	2015	SF [74]	CAC	Without	Without	Proposes to automatically select discriminative features using decision trees for better performance
16	2016	PRC [75]	ACCESS	Without	Without	Saliency fusion and progressive region classification are used to optimize depth-aware saliency models
17	2016	LBE [57]	CVPR	Without	Without	Uses a local background enclosure to capture the spread of angular directions
18	2016	SE [37]	ICME	Without	Without	Utilizes cellular automata to propagate the initial saliency map and then generate the final saliency prediction result
19	2016	DCMC [36]	SPL	Without	Without	Develops a new measure to evaluate the reliability of depth maps for reducing the influence of poor-quality depth maps on saliency detection
20	2016	BF [76]	ICPR	Without	Without	Fuses contrasting features from RGB and depth images with a Bayesian framework
21	2016	DCI [77]	ICASSP	Without	Without	Adopts the original depth map to subtract the fitted surface for generating a contrast increased map
22	2016	DSF [78]	ICASSP	Without	Without	Develops a multi-stage depth-aware saliency model for salient object detection
23	2016	GM [79]	ACCV	Without	Without	Combines color and depth-based contrast features using a generative mixture model

10.1007/s41095-020-0199-z.T002Table 2Summary of RGB-D based salient object detection methods published from 2017 to 2018

No.	Year	Method	Pub.	Training set	Backbone	Description
24	2017	HOSO [80]	DICTA	Without	Without	Combines surface orientation distribution contrast with color and depth contrast
25	2017	M $^{3}$ Net [81]	IROS	NLPR(0.65k), NJUD(1.4k)	VGG-16	Designs a multi-path multi-modal fusion strategy to integrate RGB and depth images in a task-motivated and adaptive way
26	2017	MFLN [82]	ICCVS	NLPR(0.65k), NJUD(1.4k)	AlexNet	Leverages a CNN to learn high-level representations for depth maps, and uses a multi-modal fusion network to integrate RGB and depth representations for RGB-D based salient object detection
27	2017	BED [83]	ICCVW	NLPR(0.6k), NJUD(1.2k)	GoogleNet	Uses a CNN to integrate top-down and bottom-up information for RGB-D based salient object detection, and uses a mid-level feature representation to capture background enclosure
28	2017	CDCP [84]	ICCVW	Without	Without	Proposes a novel RGB-D salient object detection algorithm using a center dark channel prior to boost performance
29	2017	TPF [85]	ICCVW	Without	Without	Leverages stereopsis to generate optical flow, which can provide an additional cue (depth cue) for producing the final detection result
30	2017	MFF [86]	SPL	Without	Without	Uses a multistage fusion framework to integrate multiple visual priors from the RGB image and depth cue for salient object detection
31	2017	MDSF [87]	TIP	NLPR(0.5k), NJUD(1.5k)	Without	Proposes a RGB-D salient object detection framework via a multi-scale discriminative saliency fusion strategy, and utilizes bootstrap learning to achieve the salient object detection task
32	2017	DF [52]	TIP	NLPR(0.75k), NJUD(1.0k)	Without	Feeds RGB and depth features into a CNN architecture to derive the saliency confidence value, and uses Laplacian propagation to produce the final detection result
33	2017	MCLP [88]	TCYB	Without	Without	Utilizes the additional depth maps and employs the existing RGB saliency map as an initialization using a refinement-cycle model to obtain the final co-saliency map
34	2018	ISC [89]	SIVP	Without	Without	Fuses salient features using both bottom-up and top-down saliency cues
35	2018	HSCS [90]	TMM	Without	Without	Utilizes a hierarchical sparsity reconstruction and energy function refinement for RGB-D based co-saliency detection
36	2018	ICS [91]	TIP	Without	Without	Exploits the constraint correlation among multiple images and introduces depth maps into the co-saliency model
37	2018	CTMF [58]	TCYB	NLPR(0.65k), NJUD(1.4k)	VGG-16	Transfers the structure of the deep color network to be applicable for the depth modality and fuses both modalities to produce the final saliency map
38	2018	PCF [92]	CVPR	NLPR(0.65k), NJUD(1.4k)	VGG-16	Designs the first multi-scale fusion architecture and a novel complementarity-aware fusion module to fuse both cross-modal and cross-level features
39	2018	SCDL [93]	ICDSP	NLPR(0.75k), NJUD(1.0k)	VGG-16	Designs a new loss function to increase the spatial coherence of salient objects
40	2018	ACCF [94]	IROS	NLPR(0.65k), NJUD(1.4k)	VGGNet	Adaptively selects complementary features from different modalities at each level, and then performs more informative cross-modal cross-level combinations
41	2018	CDB [95]	NEURO	Without	Without	Utilizes a contrast prior and depth-guided-background prior to construct a 3D stereoscopic saliency model

10.1007/s41095-020-0199-z.T003Table 3Summary of RGB-D based salient object detection models published in 2019 and 2020

No.	Year	Method	Pub.	Training set	Backbone	Description
42	2019	SSRC [96]	NEURO	NLPR(0.65k), NJUD(1.4k)	VGG-16	Uses a single-stream recurrent convolutional neural network with a four-channel input and DRCNN subnetwork
43	2019	MLF [97]	SPL	NJUD(1.588k)	VGG-16	Designs a salient object-aware data augmentation method to expand the training set
44	2019	TSRN [98]	ICIP	NJUD(1.387k)	VGG-16	Designs a fusion refinement module to integrate output features from different modalities and resolutions
45	2019	DIL [99]	MTAP	NLPR(0.5k), NJUD(0.5k)	Without	Designs a consistency integration strategy to generate an image pre-segmentation result that is consistent with the depth distribution
46	2019	CAFM [100]	TSMC	NUS [46],NCTU [101]	VGG-16	Utilizes a content-aware fusion module to integrate global and local information
47	2019	PDNet [102]	ICME	NLPR(0.5k), NJUD(1.5k)	VGG-16	Adopts a prior-model guided master network to process RGB information, which is pre-trained on the conventional RGB dataset to overcome the limited size
48	2019	MMCI [55]	PR	NLPR(0.65k), NJUD(1.4k)	VGG-16	Improves the traditional two-stream architecture by diversifying the multi-modal fusion paths and introducing cross-modal interactions in multiple layers
49	2019	TANet [103]	TIP	NLPR(0.65k), NJUD(1.4k)	VGG-16	Uses a three-stream multi-modal fusion framework to explore cross-modal complementarity in both the bottom-up and top-down processes
50	2019	DCMF [104]	TCYB	NLPR(0.65k), NJUD(1.4k)	VGG-16	Formulates a CNN-based cross-modal transfer learning problem for depth-induced salient object detection, and uses a dense cross-level feedback strategy to exploit cross-level interactions
51	2019	DGT [105]	TCYB	Without	Without	Exploits depth cues and provides a general transformation model from RGB saliency to RGB-D saliency
52	2019	LSF [45]	arXiv	NLPR(0.65k), NJUD(1.4k)	VGG	Designs an RGB-D system with three key components, including modality-specific representation learning, complementary information selection, and cross-modal complements fusion
53	2019	AFNet [106]	ACCESS	NLPR(0.65k), NJUD(1.4k)	VGG-16	Learns a switch map that is used to adaptively fuse the predicted saliency maps from the RGB and depth modality
54	2019	EPM [107]	ACCESS	Without	Without	Develops an effective propagation mechanism for RGB-D co-saliency detection
55	2019	CPFP [53]	CVPR	NLPR(0.65k), NJUD(1.4k)	VGG-16	Uses a contrast-enhanced network to obtain the one-channel enhanced map, and designs a fluid pyramid integration module to fuse cross-modal cross-level features in a pyramid style
56	2019	DMRA [54]	ICCV	NLPR(0.7k), NJUD(1.485k)	VGG-19	Designs a depth-induced multiscale recurrent attention network for salient object detection, including a depth refinement block and a recurrent attention module
57	2019	DSD [108]	JVCIR	NLPR(0.5k), NJUD(1.5k)	VGG-16	Uses a saliency fusion network to adaptively fuse both the color and depth saliency maps
58	2020	DPANet [109]	arXiv	NLPR(0.65k), NJUD(1.4k), DUT(0.8k)	ResNet-50	Uses a saliency-orientated depth perception module to evaluate the potentiality of depth maps and reduce effects of contamination
59	2020	SSDP [110]	arXiv	NLPR(0.7k), NJUD(1.485k), DUT(0.8k)	VGG-19	Makes use of existing labeled RGB saliency datasets together with unlabeled RGB-D data to boost salient object detection performance
60	2020	AttNet [111]	IVC	NLPR(0.65k), NJUD(1.4k)	VGG-16	Deploys attention maps to boost the salient objects’ location and pays more attention to the appearance information
61	2020	— [112]	NEURO	NLPR(0.65k), NJUD(1.4k)	VGG-16	Uses an adaptive gated fusion module via a GAN to obtain a better fused saliency map from RGB images and depth cues
62	2020	CoCNN [113]	PR	STERE, NJUD	VGG-16	Fuses color and disparity features from low to high layers in a unified deep model
63	2020	cmSalGAN [114]	TMM	NLPR(0.65k), NJUD(1.4k)	ResNet-50	Aims to learn an optimal view-invariant and consistent pixel-level representation for both RGB and depth images using an adversarial learning framework
64	2020	PGHF [115]	ACCESS	NLPR(0.65k), NJUD(1.4k)	VGG-16	Leverages powerful representations learned from large-scale RGB datasets to boost the model ability

10.1007/s41095-020-0199-z.T004Table 4Summary of RGB-D based salient object detection models published in 2020

No.	Year	Method	Pub.	Training set	Backbone	Description
65	2020	BiANet [116]	TIP	NLPR(0.7k), NJUD(1.485k)	VGG-16	Uses a bilateral attention module (BAM) to explore rich foreground and background information from depth maps
66	2020	ASIF-Net [117]	TCYB	NLPR(0.65k), NJUD(1.4k)	VGG-16	Integrates the attention steered complementarity from RGB-D images and introduces a global semantic constraint using adversarial learning
67	2020	Triple-Net [118]	SPL	Triple-Net	ResNe-18	Uses a triple-complementary network for RGB-D based salient object detection
68	2020	ICNet [42]	TIP	Triple-Net	VGG-16	Uses a novel information conversion module to fuse high-level RGB and depth features in an interactive and adaptive way
69	2020	SDF [119]	TIP	NLPR, NJUD,DEC, LFSD(1.5k)	VGG-16	Proposes a exemplar-driven method to estimate relatively trustworthy depth maps, and uses a selective deep saliency fusion network to effectively integrate RGB images, original depths, and newly estimated depths
70	2020	GFNet [120]	SPL	NLPR(0.8k), NJUD(1.588k)	Res2Net	Designs a gate fusion block to regularize feature fusion
71	2020	RGBS [121]	MTAP	NLPR(0.65k), NJUD(1.4k)	VGG-16	Utilizes a GAN to generate the saliency map
72	2020	D $^{3}$ Net [38]	TNNLS	NLPR(0.7k), NJUD(1.485k)	VGG-16	Uses a depth purifier unit and a three-stream feature learning module to employ low-quality depth cue filtering and cross-modal feature learning, respectively
73	2020	JL-DCF [43]	CVPR	NLPR(0.7k), NJUD(1.5k)	VGG-16, ResNet-101	Uses a joint learning strategy and a densely-cooperative fusion module to achieve better salient object detection performance
74	2020	A2dele [40]	CVPR	NLPR(0.7k), NJUD(1.485k)	VGG-16	Employs a depth distiller to explore ways of using network prediction and attention as two bridges to transfer depth knowledge to RGB images
75	2020	SSF [39]	CVPR	NLPR(0.7k), NJUD(1.485k), DUT(0.8k)	AGG-16	Designs a complimentary interaction module to select useful representations from the RGB and depth images and then integrate cross-modal features
76	2020	S $^{2}$ MA [41]	CVPR	NLPR(0.65k), NJUD(1.4k)	VGG-16	Fuses multi-modal information via self-attention and each otherâs attention strategies, and reweights the mutual attention term to filter out unreliable information
77	2020	UC-Net [44]	CVPR	NLPR(0.7k), NJUD(1.5k)	VGG-16	Uses a probabilistic RGB-D saliency detection network via a conditional VAE to generate multiple saliency maps
78	2020	CMWNet [122]	ECCV	NLPR(0.65k), NJUD(1.4k)	VGG-16	Exploits feature interactions using three cross-modal cross-scale weighting modules to improve salient object detection performance
79	2020	HDFNet [123]	ECCV	NLPR(0.7k), NJUD(1.485k), DUT(0.8k)	VGG-16	Designs a hierarchical dynamic filtering network to effectively make use of cross-modal fusion information
80	2020	CAS-GNN [124]	ECCV	NLPR(0.65k), NJUD(1.4k)	VGG-16	Designs cascaded graph neural networks to exploit useful knowledge from RGB and depth images for building powerful feature embeddings
81	2020	CMMS [125]	ECCV	NLPR(0.7k), NJUD(1.485k)	VGG-16	Proposes a cross-modality feature modulation module to enhance feature representations and an adaptive feature selection module to gradually select saliency-related features
82	2020	DANet [126]	ECCV	NLPR(0.65k), NJUD(1.4k)	VGG-16, VGG-19	Develops a single-stream network combined with a depth-enhanced dual attention to achieve real-time salient object detection
83	2020	CoNet [127]	ECCV	NLPR(0.7k), NJUD(1.485k), DUT(0.8k)	ResNet	Develops a collaborative learning framework for RGB-D based salient object detection. Three collaborators (edge detection, coarse salient object detection and depth estimation) are utilized to jointly boost the performance
84	2020	BBS-Net [128]	ECCV	NLPR(0.65k), NJUD(1.4k)	VGG-16, VGG-19, ResNet-50	Uses a bifurcated backbone strategy to learn teacher and student features, and utilizes a depth-enhanced module to excavate informative parts of depth cues
85	2020	ATSA [129]	ECCV	NLPR(0.7k), NJUD(1.485k), DUT(0.8k)	VGG-19	Proposes an asymmetric two-stream architecture taking account of the inherent differences between RGB and depth data for salient object detection
86	2020	PGAR [130]	ECCV	NLPR(0.7k), NJUD(1.485k)	VGG-16	Proposes a progressively guided alternate refinement network to produce a coarse initial prediction using a multi-scale residual block
87	2020	MCINet [131]	arXiv	NLPR(0.65k), NJUD(1.4k)	ResNet-50	Develops a novel multi-level cross-modal interaction network for RGB-D salient object detection
88	2020	DRLF [132]	TIP	NLPR(0.65k), NJUD(1.4k)	VGG-16	Develops a channel-wise fusion network to conduct multi-net and multi-level selective fusion for RGB-D salient object detection
89	2020	DQAM [133]	arXiv	NLPR(0.65k), NJUD(1.4k)	Without	Proposes a depth quality assessment solution to conduct “quality-aware” salient object detection for RGB-D images
90	2020	DQSD [134]	TIP	NLPR(0.65k), NJUD(1.4k)	VGG-19	Integrates a depth quality aware subnet into a bi-stream structure to assess the depth quality before conducting RGB-D fusion
91	2020	DASNet [135]	ACM MM	NLPR(0.7k), NJUD(1.5k)	ResNet-50	Proposes a new perspective of containing the depth constraints in the learning process rather than using depths as inputs
92	2020	DCMF [136]	TIP	NLPR(0.65k), NJUD(1.4k)	VGG-16, ResNet-50	Designs a disentangled cross-modal fusion network to expose structural and content representations from RGB and depth images

10.1007/s41095-020-0199-z.T005Table 5RGB-D based salient object detection models with open-source implementations

Year	Model	Implementation	Code link
2014	LHM [51]	Matlab	https://sites.google.com/site/rgbdsaliency/code
2014	DESM [49]	Matlab	https://github.com/HzFu/DES_code
2015	GP [50]	Matlab	https://github.com/JianqiangRen/Global_Priors_RGBD_Saliency_Detection
2016	DCMC [36]	Matlab	https://github.com/rmcong/Code-for-DCMC-method
2016	LBE [57]	Matlab & C++	http://users.cecs.anu.edu.au/ u4673113/lbe.html
2017	BED [83]	Caffe	https://github.com/sshige/rgbd-saliency
	CDCP [84]	Matlab	https://github.com/ChunbiaoZhu/ACVR2017
	MDSF [87]	Matlab	https://github.com/ivpshuhttps://github.com/ivpshu
	DF [52]	Matlab	https://pan.baidu.com/s/1Y-PqAjuH9xREBjfl7H45HA
2018	CTMF [58]	Caffe	https://github.com/haochen593/CTMF
	PCF [92]	Caffe	https://github.com/haochen593/PCA-Fuse_RGBD_CVPR18
	PDNet [102]	TensorFlow	https://github.com/cai199626/PDNet
2019	AFNet [106]	TensorFlow	https://github.com/Lucia-Ningning/Adaptive_Fusion_RGBD_Saliency_Detection
	CPFP [53]	Caffe	https://github.com/JXingZhao/ContrastPrior
	DMRA [54]	PyTorch	https://github.com/jiwei0921/DMRA
	DGT [105]	Matlab	https://github.com/rmcong/Code-for-DTM-Method
2020	ICNet [42]	Caffe	https://github.com/MathLee/ICNet-for-RGBD-SOD
	JL-DCF [43]	Pytorch, Caffe	https://github.com/kerenfu/JLDCF
	A2dele [40]	PyTorch	https://github.com/OIPLab-DUT/CVPR2020-A2dele
	SSF [39]	PyTorch	https://github.com/OIPLab-DUT/CVPR_SSF-RGBD
	ASIF-Net [117]	TensorFlow	https://github.com/Li-Chongyi/ASIF-Net
	S $^{2}$ MA [41]	PyTorch	https://github.com/nnizhang/S2MA
	UC-Net [44]	PyTorch	https://github.com/JingZhang617/UCNet
	D $^{3}$ Net [38]	PyTorch	https://github.com/DengPingFan/D3NetBenchmark
	CMWNet [122]	Caffe	https://github.com/MathLee/CMWNet
	HDFNet [123]	PyTorch	https://github.com/lartpang/HDFNet
	CMMS [125]	TensorFlow	https://github.com/Li-Chongyi/cmMS-ECCV20
	CAS-GNN [124]	PyTorch	https://github.com/LA30/Cas-Gnn
	DANet [126]	PyTorch	https://github.com/Xiaoqi-Zhao-DLUT/DANet-RGBD-Saliency
	CoNet [127]	PyTorch	https://github.com/jiwei0921/CoNet
	DASNet [135]	PyTorch	http://cvteam.net/projects/2020/DASNet/
	BBS-Net [128]	PyTorch	https://github.com/DengPingFan/BBS-Net
	ATSA [129]	PyTorch	https://github.com/sxfduter/ATSA
	PGAR [130]	PyTorch	https://github.com/ShuhanChen/PGAR_ECCV20
	FRDT [138]	PyTorch	https://github.com/jack-admiral/ACM-MM-FRDT

10.1007/s41095-020-0199-z.T006Table 6Nine RGB-D benchmark datasets, by year, place of publication (Pub.), dataset size, number of objects in the images (#Obj.), type of scene, depth sensor, and resolution. They can be downloaded from our website: http://dpfan.net/d3netbenchmark/

No.	Dataset	Year	Pub.	Size	#Obj.	Types	Sensor	Resolution
1	STERE [139]	2012	CVPR	1000	$\sim$ One	Internet	Stereo camera+sift flow	$[251 - 1200] \times [222 - 900]$
2	GIT [47]	2013	BMVC	80	Multiple	Home environment	Microsoft Kinect	$640 \times 480$
3	DES [49]	2014	ICIMCS	135	One	Indoor	Microsoft Kinect	$640 \times 480$
4	NLPR [51]	2014	ECCV	1000	Multiple	Indoor/outdoor	Microsoft Kinect	$640 \times 480$ , $480 \times 640$
5	LFSD [140]	2014	CVPR	100	One	Indoor/outdoor	Lytro Illum camera	$360 \times 360$
6	NJUD [56]	2014	ICIP	1985	$\sim$ One	Movie/Internet/photo	FujiW3 camera+optical flow	$[231 - 1213] \times [274 - 828]$
7	SSD [85]	2017	ICCVW	80	Multiple	Movies	Sun’s optical flow	$960 \times 1080$
8	DUT-RGBD [137]	2019	ICCV	1200	Multiple	Indoor/outdoor	—	$400 \times 600$
9	SIP [38]	2020	TNNLS	929	Multiple	Person in the wild	Huawei Mate10	$992 \times 744$

10.1007/s41095-020-0199-z.T007Table 7Popular light field salient object detection methods

No.	Year	Method	Pub.	Dataset	Description
1	2014	LFS [140]	CVPR	LFSD	The first light-field saliency detection algorithm employs object identity and focus cues based on the refocusing capability of the light field
2	2015	WSC [142]	CVPR	LFSD	Uses a weighted sparse coding framework to learn a saliency/non-saliency dictionary
3	2015	DILF [143]	IJCAI	LFSD	Incorporates depth contrast to complement the disadvantage of color and uses focus-based background priors to boost the saliency detection performance
4	2016	RL [144]	ICASSP	LFSD	Utilizes the inherent structure information in light field images to improve saliency detection
5	2017	MA [145]	TOMM	HFUT, LFSD	Integrates multiple saliency cues extracted from light field images using a random-search-based weighting strategy
6	2017	BIF [146]	NPL	LFSD	Integrates color-based contrast, depth-induced contrast, focus map of foreground slice, and background weighted depth contrast using a two-stage Bayesian integration framework
7	2017	LFS [147]	TPAMI	LFSD	An extension of Ref. [140]
8	2017	RLM [148]	ICIVC	LFSD	Utilizes the light field relative location measurement for salient object detection on light field images
9	2018	SGDC [149]	CVPR	LFSD	Designs a saliency-guided depth optimization framework for multi-layer light field displays
10	2018	DCA [150]	FiO	LFSD	Proposes a graph model depth-induced cellular automata to optimize saliency maps using light field data
11	2019	DLLF [151]	ICCV	DUTLF-FS, LFSD	Utilizes a recurrent attention network to fuse each slice from the focal stack to learn the most informative features
12	2019	DLSD [152]	IJCAI	DUTLF-MV	Formulates saliency detection into two subproblems, including 1) light field synthesis from a single view and 2) light-field-driven saliency detection
13	2019	Molf [153]	NIPS	UTLF-FS	Uses a memory-oriented decoder for light field salient object detection
14	2020	ERNet [154]	AAAI	DUTLF-FS, HFUT, LFSD	Uses an asymmetrical two-stream architecture to overcome computation-intensive and memory-intensive challenges in a high-dimensional light field data
15	2020	DCA [137]	TIP	LFSD	Presents a saliency detection framework on light fields based on the depth-induced cellular automata (DCA) model. It can enforce spatial consistency to optimize the inaccurate saliency map using the DCA model
16	2020	RDFD [155]	MTAP	LFSD	Defines a region-based depth feature descriptor extracted from the light field focal stack to facilitate low- and high-level cues for saliency detection
17	2020	LFNet [141]	TIP	DUTLF-FS, LFSD, HFUT	Utilizes a light field refinement module and a light field integration module to effectively integrate multiple cues (focus, depth, and object identity) from light field images
18	2020	LFDCN [156]	TIP	Lytro Illum, LFSD, HFUT	Uses a deep convolutional network based on the modified DeepLab-v2 model to explore spatial and multi-view properties of light field images for saliency detection

10.1007/s41095-020-0199-z.T008Table 8Attribute-based study w.r.t. salient object scales. 24 representative RGB-D based salient object detection models (9 traditional, 15 deep learning-based) are compared in terms of MAE and $S_{α}$ . The three best results are shown in red, blue, and green

		Traditional models									Deep learning-based models
	Scale	LHM [51]	ACSD [56]	DESM [49]	GP [50]	LBE [57]	DCMC [36]	SE [37]	CDCP [84]	CDB [95]	DF [52]	PCF [92]	CTMF [58]	CPFP [53]	TANet [103]	AFNet [106]	MMCI [55]	DMRA [54]	D $^{3}$ Net [38]	SSF [39]	A2dele [40]	S $^{2}$ MA [41]	ICNet [42]	JL-DCF [43]	UC-Net [44]
MAE	Small	0.065	0.149	0.319	0.098	0.177	0.108	0.056	0.128	0.073	0.087	0.042	0.065	0.044	0.041	0.046	0.051	0.030	0.033	0.031	0.032	0.035	0.036	0.032	0.034
	Medium	0.178	0.183	0.287	0.180	0.210	0.158	0.150	0.173	0.179	0.152	0.068	0.107	0.055	0.067	0.095	0.079	0.069	0.053	0.045	0.054	0.052	0.052	0.041	0.042
	Large	0.403	0.311	0.310	0.377	0.261	0.305	0.364	0.308	0.385	0.310	0.112	0.183	0.093	0.118	0.213	0.130	0.181	0.102	0.105	0.114	0.088	0.104	0.085	0.072
	Overall	0.166	0.184	0.296	0.173	0.206	0.156	0.142	0.171	0.167	0.147	0.065	0.102	0.055	0.065	0.091	0.076	0.067	0.052	0.046	0.053	0.051	0.052	0.041	0.042
$S_{α}$	Small	0.624	0.668	0.517	0.650	0.645	0.700	0.775	0.661	0.666	0.745	0.847	0.789	0.840	0.846	0.792	0.832	0.860	0.879	0.876	0.859	0.877	0.882	0.881	0.883
	Medium	0.543	0.732	0.658	0.598	0.723	0.727	0.676	0.683	0.585	0.730	0.863	0.805	0.877	0.862	0.779	0.859	0.838	0.888	0.893	0.865	0.893	0.892	0.906	0.901
	Large	0.386	0.630	0.686	0.450	0.731	0.604	0.479	0.586	0.424	0.597	0.838	0.761	0.855	0.827	0.682	0.830	0.734	0.846	0.837	0.815	0.863	0.845	0.859	0.876
	Overall	0.552	0.710	0.626	0.601	0.705	0.712	0.686	0.671	0.593	0.725	0.857	0.798	0.867	0.856	0.776	0.851	0.836	0.883	0.885	0.860	0.887	0.886	0.897	0.895

10.1007/s41095-020-0199-z.T009Table 9Attribute-based study w.r.t. background clutter. 24 representative RGB-D based salient object detection models (9 traditional, 15 deep learning-based) are compared in terms of MAE and $S_{α}$ . The three best results are shown in red, blue, and green

		Traditional models									Deep learning-based models
	background	LHM [51]	ACSD [56]	DESM [49]	GP [50]	LBE [57]	DCMC [36]	SE [37]	CDCP [84]	CDB [95]	DF [52]	PCF [92]	CTMF [58]	CPFP [53]	TANet [103]	AFNet [106]	MMCI [55]	DMRA [54]	D $^{3}$ Net [38]	SSF [39]	A2dele [40]	S $^{2}$ MA [41]	ICNet [42]	JL-DCF [43]	UC-Net [44]
MAE	Simple	0.100	0.163	0.219	0.150	0.202	0.056	0.084	0.028	0.136	0.045	0.031	0.053	0.018	0.033	0.031	0.041	0.028	0.017	0.012	0.010	0.016	0.013	0.014	0.013
	Uncertain	0.164	0.195	0.294	0.175	0.210	0.140	0.133	0.139	0.159	0.129	0.062	0.081	0.050	0.059	0.075	0.070	0.058	0.045	0.043	0.043	0.049	0.041	0.037	0.037
	Complex	0.159	0.190	0.349	0.180	0.205	0.190	0.147	0.236	0.143	0.163	0.085	0.110	0.079	0.077	0.108	0.094	0.087	0.071	0.065	0.070	0.072	0.079	0.063	0.065
	Overall	0.160	0.193	0.295	0.174	0.209	0.140	0.132	0.141	0.157	0.127	0.063	0.082	0.051	0.059	0.076	0.070	0.059	0.046	0.043	0.043	0.049	0.043	0.038	0.038
$S_{α}$	Simple	0.781	0.787	0.761	0.694	0.748	0.930	0.856	0.941	0.704	0.944	0.944	0.913	0.958	0.937	0.922	0.933	0.935	0.960	0.966	0.965	0.965	0.969	0.961	0.962
	Uncertain	0.572	0.694	0.638	0.606	0.695	0.736	0.723	0.727	0.610	0.774	0.873	0.853	0.882	0.873	0.818	0.868	0.854	0.900	0.894	0.884	0.895	0.910	0.909	0.907
	Complex	0.496	0.627	0.509	0.545	0.616	0.577	0.605	0.487	0.575	0.627	0.782	0.742	0.787	0.790	0.694	0.768	0.751	0.822	0.815	0.786	0.813	0.808	0.829	0.833
	Overall	0.576	0.693	0.633	0.606	0.691	0.732	0.720	0.718	0.612	0.770	0.869	0.847	0.878	0.869	0.813	0.863	0.850	0.896	0.891	0.879	0.892	0.904	0.904	0.904

10.1007/s41095-020-0199-z.T010Table 10Attribute-based study w.r.t. background objects: car, barrier, flower, grass, road, sign, tree, and other. The methods compared including 24 representative RGB-D based salient object detection models (9 traditional and 15 deep learning-based) evaluated on the SIP dataset [38] in terms of MAE and $S_{α}$ . The three best results are shown in red, blue, and green

		Traditional models									Deep learning-based models
	Categories	LHM [51]	ACSD [56]	DESM [49]	GP [50]	LBE [57]	DCMC [36]	SE [37]	CDCP [84]	CDB [95]	DF [52]	PCF [92]	CTMF [58]	CPFP [53]	TANet [103]	AFNet [106]	MMCI [55]	DMRA [54]	D $^{3}$ Net [38]	SSF [39]	A2dele [40]	S $^{2}$ MA [41]	ICNet [42]	JL-DCF [43]	UC-Net [44]
MAE	Car	0.158	0.163	0.301	0.159	0.201	0.185	0.154	0.202	0.171	0.171	0.085	0.134	0.094	0.084	0.101	0.093	0.069	0.061	0.063	0.078	0.055	0.067	0.058	0.057
	Barrier	0.197	0.177	0.308	0.180	0.201	0.196	0.176	0.251	0.203	0.202	0.073	0.149	0.060	0.078	0.128	0.089	0.093	0.068	0.054	0.074	0.057	0.075	0.052	0.053
	Flower	0.105	0.122	0.306	0.099	0.186	0.158	0.063	0.141	0.101	0.132	0.091	0.075	0.133	0.100	0.090	0.081	0.046	0.095	0.107	0.051	0.104	0.025	0.054	0.075
	Grass	0.164	0.161	0.279	0.155	0.184	0.167	0.138	0.182	0.176	0.167	0.041	0.110	0.035	0.048	0.088	0.059	0.056	0.037	0.030	0.046	0.033	0.043	0.023	0.029
	Road	0.189	0.167	0.281	0.176	0.187	0.181	0.164	0.225	0.189	0.169	0.070	0.140	0.054	0.072	0.125	0.078	0.093	0.059	0.049	0.072	0.050	0.065	0.045	0.044
	Sign	0.107	0.126	0.268	0.110	0.184	0.126	0.079	0.134	0.118	0.096	0.058	0.101	0.063	0.060	0.077	0.083	0.051	0.055	0.051	0.054	0.048	0.054	0.050	0.057
	Tree	0.192	0.193	0.310	0.190	0.241	0.194	0.183	0.230	0.219	0.205	0.083	0.157	0.083	0.091	0.132	0.109	0.106	0.083	0.067	0.074	0.092	0.097	0.063	0.071
	Other	0.246	0.217	0.329	0.224	0.229	0.216	0.229	0.274	0.233	0.233	0.106	0.177	0.111	0.111	0.170	0.124	0.140	0.095	0.083	0.099	0.100	0.100	0.084	0.086
	Overall	0.184	0.172	0.298	0.173	0.200	0.186	0.164	0.224	0.192	0.185	0.071	0.139	0.064	0.075	0.118	0.086	0.085	0.063	0.053	0.070	0.057	0.069	0.049	0.051
$S_{α}$	Car	0.516	0.731	0.590	0.603	0.714	0.671	0.591	0.613	0.546	0.631	0.811	0.726	0.786	0.807	0.736	0.813	0.817	0.856	0.845	0.804	0.870	0.846	0.855	0.859
	Barrier	0.497	0.727	0.609	0.575	0.728	0.672	0.612	0.553	0.552	0.643	0.837	0.698	0.860	0.831	0.708	0.830	0.792	0.855	0.874	0.821	0.871	0.848	0.876	0.875
	Flower	0.477	0.775	0.573	0.673	0.703	0.707	0.772	0.667	0.639	0.750	0.771	0.738	0.714	0.760	0.688	0.785	0.824	0.789	0.768	0.845	0.804	0.901	0.856	0.811
	Grass	0.537	0.756	0.643	0.605	0.760	0.728	0.683	0.672	0.559	0.672	0.908	0.770	0.908	0.899	0.780	0.888	0.876	0.917	0.924	0.878	0.928	0.910	0.939	0.924
	Road	0.521	0.739	0.634	0.598	0.751	0.685	0.641	0.595	0.576	0.680	0.851	0.722	0.871	0.848	0.705	0.847	0.807	0.873	0.885	0.832	0.885	0.868	0.889	0.892
	Sign	0.578	0.786	0.634	0.628	0.719	0.745	0.761	0.714	0.615	0.757	0.855	0.756	0.833	0.857	0.771	0.818	0.848	0.849	0.849	0.842	0.871	0.861	0.859	0.840
	Tree	0.505	0.699	0.606	0.577	0.661	0.648	0.600	0.588	0.543	0.625	0.802	0.679	0.804	0.778	0.691	0.779	0.748	0.806	0.837	0.807	0.800	0.788	0.848	0.825
	Other	0.460	0.687	0.594	0.532	0.706	0.669	0.563	0.554	0.542	0.600	0.786	0.677	0.774	0.782	0.647	0.790	0.722	0.800	0.828	0.785	0.809	0.799	0.821	0.823
	Overall	0.511	0.732	0.616	0.588	0.727	0.683	0.628	0.595	0.557	0.653	0.842	0.716	0.850	0.835	0.720	0.833	0.806	0.860	0.874	0.828	0.872	0.854	0.880	0.875

10.1007/s41095-020-0199-z.T011Table 11Attribute-based study w.r.t. light conditions (sunny vs. low-light). The comparison methods include 24 representative RGB-D based salient object detection models (9 traditional models and 15 deep learning-based models) evaluated on the SIP dataset [38] in terms of MAE and $S_{α}$ . The three best results are shown in $red$ , $blue$ , and $green$ fonts

		Traditional models									Deep learning-based models
	Conditions	LHM [51]	ACSD [56]	DESM [49]	GP [50]	LBE [57]	DCMC [36]	SE [37]	CDCP [84]	CDB [95]	DF [52]	PCF [92]	CTMF [58]	CPFP [53]	TANet [103]	AFNet [106]	MMCI [55]	DMRA [54]	D $^{3}$ Net [38]	SSF [39]	A2dele [40]	S $^{2}$ MA [41]	ICNet [42]	JL-DCF [43]	UC-Net [44]
MAE	Sunny	0.182	0.171	0.294	0.171	0.200	0.183	0.160	0.218	0.190	0.181	0.069	0.137	0.062	0.075	0.116	0.085	0.083	0.062	0.052	0.068	0.057	0.068	0.048	0.051
	Low-light	0.198	0.178	0.323	0.187	0.201	0.207	0.193	0.268	0.208	0.211	0.078	0.154	0.073	0.076	0.130	0.091	0.103	0.067	0.059	0.080	0.058	0.081	0.059	0.055
	Overall	0.184	0.172	0.298	0.173	0.200	0.186	0.164	0.224	0.192	0.185	0.071	0.139	0.064	0.075	0.118	0.086	0.085	0.063	0.053	0.070	0.057	0.069	0.049	0.051
$S_{α}$	Sunny	0.516	0.733	0.622	0.593	0.728	0.690	0.639	0.607	0.560	0.660	0.843	0.718	0.852	0.834	0.723	0.833	0.811	0.861	0.875	0.831	0.872	0.856	0.882	0.876
	low-light	0.481	0.721	0.573	0.554	0.722	0.635	0.556	0.515	0.543	0.610	0.838	0.701	0.838	0.837	0.700	0.832	0.775	0.855	0.867	0.810	0.871	0.839	0.867	0.871
	Overall	0.511	0.732	0.616	0.588	0.727	0.683	0.628	0.595	0.557	0.653	0.842	0.716	0.850	0.835	0.720	0.833	0.806	0.860	0.874	0.828	0.872	0.854	0.880	0.875