AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Computer Science and Technology Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Regular Paper

Learning Local Contrast for Crisp Edge Detection

Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
Show Author Information

Abstract

In recent years, the accuracy of edge detection on several benchmarks has been significantly improved by deep learning based methods. However, the prediction of deep neural networks is usually blurry and needs further post-processing including non-maximum suppression and morphological thinning. In this paper, we demonstrate that the blurry effect arises from the binary cross-entropy loss, and crisp edges could be obtained directly from deep convolutional neural networks. We propose to learn edge maps as the representation of local contrast with a novel local contrast loss. The local contrast is optimized in a stochastic way to focus on specific edge directions. Experiments show that the edge detection network trained with local contrast loss achieves a high accuracy comparable to previous methods and dramatically improves the crispness. We also present several applications of the crisp edges, including image completion, image retrieval, sketch generation, and video stylization.

Keywords

edge detectionimage processingneural networklocal contrast

Electronic Supplementary Material

Download File(s)
JCST-2301-13101-Highlights.pdf (122.4 KB)

References

[1]

Kittler J. On the accuracy of the sobel edge detector. Image and Vision Computing, 1983, 1(1): 37–42. DOI: 10.1016/0262-8856(83)90006-9.
Crossref Google Scholar
[2]

Canny J. A computational approach to edge detection. IEEE Trans. Pattern Analysis and Machine Intelligence, 1986, PAMI-8(6): 679–698. DOI: 10.1109/TPAMI.1986.4767851.
Crossref Google Scholar
[3]

Konishi S, Yuille A L, Coughlan J M, Zhu S C. Statistical edge detection: Learning and evaluating edge cues. IEEE Trans. Pattern Analysis and Machine Intelligence, 2003, 25(1): 57–74. DOI: 10.1109/TPAMI.2003.1159946.
Crossref Google Scholar
[4]

Martin D R, Fowlkes C C, Malik J. Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Analysis and Machine Intelligence, 2004, 26(5): 530–549. DOI: 10.1109/TPAMI.2004.1273918.
Crossref Google Scholar
[5]
Paris S. Edge-preserving smoothing and mean-shift segmentation of video streams. In Proc. the 10th European Conference on Computer Vision: Part II, Oct. 2008, pp.460–473. DOI: 10.1007/978-3-540-88688-4_34.
Crossref
[6]
Ren X F, Bo L F. Discriminatively trained sparse code gradients for contour detection. In Proc. the 25th International Conference on Neural Information Processing Systems, Dec. 2012, pp.584–592. DOI: 10.5555/2999134.2999200.
[7]
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arxiv: 1409.1556, 2015. DOI: 10.48550/arXiv.1409.1556.
[8]
He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.770–778. DOI: 10.1109/CVPR.2016.90.
Crossref
[9]
Girshick R. Fast R-CNN. In Proc. the 2015 IEEE International Conference on Computer Vision, Dec. 2015, pp.1440–1448. DOI: 10.1109/ICCV.2015.169.
Crossref
[10]

Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149. DOI: 10.1109/TPAMI.2016.2577031.
Crossref Google Scholar
[11]

Chen L C, Papandreou G, Kokkinos I, Murphy K, Yuille A L. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Analysis and Machine Intelligence, 2018, 40(4): 834–848. DOI: 10.1109/TPAMI.2017.2699184.
Crossref Google Scholar
[12]
He K M, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In Proc. the 2017 IEEE International Conference on Computer Vision, Oct. 2017, pp.2980–2988. DOI: 10.1109/ICCV.2017.322.
Crossref
[13]
Xie S N, Tu Z W. Holistically-nested edge detection. In Proc. the 2015 IEEE International Conference on Computer Vision, Dec. 2015, pp.1395–1403. DOI: 10.1109/ICCV.2015.164.
Crossref
[14]
Maninis K K, Pont-Tuset J, Arbeláez P, Van Gool L. Convolutional oriented boundaries. In Proc. the 14th European Conference on Computer Vision, Sept. 2016, pp.580–596. DOI: 10.1007/978-3-319-46448-0_35.
Crossref
[15]
Liu Y, Cheng M M, Hu X W, Wang K, Bai X. Richer convolutional features for edge detection. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.5872–5881. DOI: 10.1109/CVPR.2017.622.
Crossref
[16]
He J Z, Zhang S L, Yang M, Shan Y H, Huang T J. Bi-directional cascade network for perceptual edge detection. In Proc. the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2019, pp.3823–3832. DOI: 10.1109/CVPR.2019.00395.
Crossref
[17]

Arbelaez P, Maire M, Fowlkes C, Malik J. Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Analysis and Machine Intelligence, 2011, 33(5): 898–916. DOI: 10.1109/TPAMI.2010.161.
Crossref Google Scholar
[18]
Silberman N, Hoiem D, Kohli P, Fergus R. Indoor segmentation and support inference from RGBD images. In Proc. the 12th European Conference on Computer Vision, Oct. 2012, pp.746–760. DOI: 10.1007/978-3-642-33715-4_54.
Crossref
[19]

Mély D A, Kim J, McGill M, Guo Y L, Serre T. A systematic comparison between visual cues for boundary detection. Vision Research, 2016, 120: 93–107. DOI: 10.1016/j.visres.2015.11.007.
Crossref Google Scholar
[20]

Huan L X, Xue N, Zheng X W, He W, Gong J Y, Xia G S. Unmixing convolutional features for crisp edge detection. IEEE Trans. Pattern Analysis and Machine Intelligence, 2022, 44(10): 6602–6609. DOI: 10.1109/TPAMI.2021.3084197.
Crossref Google Scholar
[21]
Wang Y P, Zhao X, Huang K Q. Deep crisp boundaries. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.1724–1732. DOI: 10.1109/CVPR.2017.187.
Crossref
[22]
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In Proc. the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention, Oct. 2015, pp.234–241. DOI: 10.1007/978-3-319-24574-4_28.
Crossref
[23]
Deng R X, Shen C H, Liu S J, Wang H B, Liu X R. Learning to predict crisp boundaries. In Proc. the 15th European Conference on Computer Vision, Sept. 2018, pp.570–586. DOI: 10.1007/978-3-030-01231-1_35.
Crossref
[24]

Wang M, Fang X N, Yang G W, Shamir A, Hu S M. Prominent structures for video analysis and editing. IEEE Trans. Visualization and Computer Graphics, 2021, 27(7): 3305–3317. DOI: 10.1109/TVCG.2020.2970045.
Crossref Google Scholar
[25]
Nazeri K, Ng E, Joseph T, Qureshi F Z, Ebrahimi M. EdgeConnect: Generative image inpainting with adversarial edge learning. arXiv: 1901.00212, 2019. https://arxiv.org/abs/1901.00212, May 2023.
[26]
Lu P, Huang G, Lin H Y, Yang W M, Guo G D, Fu Y W. Domain-aware se network for sketch-based image retrieval with multiplicative Euclidean margin softmax. In Proc. the 29th ACM International Conference on Multimedia, Oct. 2021, pp.3418–3426. DOI: 10.1145/3474085.3475499.
Crossref
[27]
Li M T, Lin Z, Mech R, Yumer E, Ramanan D. Photo-sketching: Inferring contour drawings from images. In Proc. the 2019 IEEE Winter Conference on Applications of Computer Vision, Jan. 2019, pp.1403–1412. DOI: 10.1109/WACV.2019.00154.
Crossref
[28]

Winnemöller H, Olsen S C, Gooch B. Real-time video abstraction. ACM Trans. Graphics, 2006, 25(3): 1221–1226. DOI: 10.1145/1141911.1142018.
Crossref Google Scholar
[29]

Torre V, Poggio T A. On edge detection. IEEE Trans. Pattern Analysis and Machine Intelligence, 1986, PAMI-8(2): 147–163. DOI: 10.1109/TPAMI.1986.4767769.
Crossref Google Scholar
[30]
Deng J, Dong W, Socher R, Li L J, Li K, Li F F. ImageNet: A large-scale hierarchical image database. In Proc. the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2009, pp.248–255. DOI: 10.1109/CVPR.2009.5206848.
Crossref
[31]

Liu J J, Hou Q B, Cheng M M. Dynamic feature integration for simultaneous detection of salient object, edge, and skeleton. IEEE Trans. Image Processing, 2020, 29: 8652–8667. DOI: 10.1109/TIP.2020.3017352.
Crossref Google Scholar
[32]
Soria X, Riba E, Sappa A. Dense extreme inception network: Towards a robust CNN model for edge detection. In Proc. the 2020 IEEE Winter Conference on Applications of Computer Vision, Mar. 2020, pp.1923–1932. DOI: 10.1109/WACV45572.2020.9093290.
Crossref
[33]

Mittal M, Verma A, Kaur I, Kaur B, Sharma M, Goyal L M, Roy S, Kim T H. An efficient edge detection approach to provide better edge connectivity for image analysis. IEEE Access, 2019, 7: 33240–33255. DOI: 10.1109/ACCESS.2019.2902579.
Crossref Google Scholar
[34]
Hallman S, Fowlkes C C. Oriented edge forests for boundary detection. In Proc. the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.1732–1740. DOI: 10.1109/CVPR.2015.7298782.
Crossref
[35]
Ganin Y, Lempitsky V. N4-fields: Neural network nearest neighbor fields for image transforms. In Proc. the 2014 Asian Conference on Computer Vision, Apr. 2014, pp.536–551. DOI: 10.1007/978-3-319-16808-1_36.
Crossref
[36]
Shen W, Wang X G, Wang Y, Bai X, Zhang Z J. Deepcontour: A deep convolutional feature learned by positive-sharing loss for contour detection. In Proc. the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.3982–3991. DOI: 10.1109/CVPR.2015.7299024.
Crossref
[37]
Bertasius G, Shi J B, Torresani L. High-for-low and low-for-high: Efficient boundary detection from deep object features and its applications to high-level vision. In Proc. the 2015 IEEE International Conference on Computer Vision, Dec. 2015, pp.504–512. DOI: 10.1109/ICCV.2015.65.
Crossref
[38]
Yang J M, Price B, Cohen S, Lee H, Yang M H. Object contour detection with a fully convolutional encoder-decoder network. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.193–202. DOI: 10.1109/CVPR.2016.28.
Crossref
[39]
Kokkinos I. Pushing the boundaries of boundary detection using deep learning. arXiv: 1511.07386, 2015. https://arxiv.org/pdf/1511.07386.pdf, May 2023.
[40]
Xu D, Ouyang W L, Alameda-Pineda X, Ricci E, Wang X G, Sebe N. Learning deep structured multi-scale features using attention-gated CRFs for contour prediction. In Proc. the 31st International Conference on Neural Information Processing Systems, Dec. 2017, pp.3964–3973. DOI: 10.5555/3294996.3295152.
[41]
Liao Y, Fu S P, Lu X Q, Zhang C C, Tang Z. Deep-learning-based object-level contour detection with CCG and CRF optimization. In Proc. the 2017 IEEE International Conference on Multimedia and Expo, Jul. 2017, pp.859–864. DOI: 10.1109/ICME.2017.8019358.
Crossref
[42]
Liu Z W, Luo P, Wang X G, Tang X O. Deep learning face attributes in the wild. In Proc. the 2015 IEEE International Conference on Computer Vision, Dec. 2015, pp.3730–3738. DOI: 10.1109/ICCV.2015.425.
Crossref
[43]

Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Processing, 2004, 13(4): 600–612. DOI: 10.1109/TIP.2003.819861.
Crossref Google Scholar
[44]
Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Proc. the 31st International Conference on Neural Information Processing Systems, Dec. 2017, pp.6629–6640. DOI: 10.5555/3295222.3295408.
[45]
Eitz M, Hays J, Alexa M. How do humans sketch objects? ACM Trans. Graphics, 2012, 31(4): Article No. 44. DOI: 10.1145/2185520.2185540.
Crossref
Journal of Computer Science and Technology
Volume 38 Issue 3,
May 2023
Pages 554-566
DOI: 10.1007/s11390-023-3101-5
Cite this article:
Fang X-N, Zhang S-H. Learning Local Contrast for Crisp Edge Detection. Journal of Computer Science and Technology, 2023, 38(3): 554-566. https://doi.org/10.1007/s11390-023-3101-5

366

Views

1

Crossref

1

Web of Science

1

Scopus

0

CSCD

Altmetrics
Received: 16 January 2023
Accepted: 15 May 2023
Published: 30 May 2023
© Institute of Computing Technology, Chinese Academy of Sciences 2023
Return