AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Computational Visual Media Article
PDF (4.2 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Towards robustness and generalization of point cloud representation: A geometry coding method and a large-scale object-level dataset

Guangdong–Hong Kong–Macao Joint Laboratory of Human–Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology, Chinese Academyof Sciences, Shenzhen 518000, China
University of Chinese Academy of Sciences, Beijing 065001, China
Shanghai AI Laboratory, Shanghai 200001, China
SIAT Branch, Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen 518000, China
Alibaba DAMO Academy, Hangzhou 242332, China
Show Author Information

Graphical Abstract

Abstract

Robustness and generalization are two challenging problems for learning point cloud represen-tation. To tackle these problems, we first design a novel geometry coding model, which can effectively use an invariant eigengraph to group points with similar geometric information, even when such points are far from each other. We also introduce a large-scale point cloud dataset, PCNet184. It consists of 184 categories and 51,915 synthetic objects, which brings new challenges for point cloud classification, and provides a new benchmark to assess point cloud cross-domain generalization. Finally, we perform exten-sive experiments on point cloud classification, using ModelNet40, ScanObjectNN, and our PCNet184, and segmentation, using ShapeNetPart and S3DIS. Our method achieves comparable performance to state-of-the-art methods on these datasets, for both supervised and unsupervised learning. Code and our dataset are available at https://github.com/MingyeXu/PCNet184.

Keywords

geometry codingself-supervised learningpoint cloudclassificationsegmentation3D analysis

References

[1]
Alqazzaz, S.; Sun, X. F.; Yang, X.; Nokes, L. Automated brain tumor segmentation on multi-modal MR image using SegNet. Computational Visual Media Vol. 5, No. 2, 209–219, 2019.
Crossref Google Scholar
[2]
Han, W. K.; Wu, H.; Wen, C. L.; Wang, C.; Li, X. BLNet: Bidirectional learning network for point clouds. Computational Visual Media Vol. 8, No. 4, 585–596, 2022.
Crossref Google Scholar
[3]
Zhang, J. H.; Wang, Y. L.; Zhou, Z. P.; Luan, T. Y.; Wang, Z.; Qiao, Y. Learning dynamical human-joint affinity for 3D pose estimation in videos. IEEE Transactions on Image Processing Vol. 30, 7914–7925, 2021.
Crossref Google Scholar
[4]
Wu, Z. R.; Song, S. R.; Khosla, A.; Yu, F.; Zhang, L. G.; Tang, X. O.; Xiao, J. X. 3D ShapeNets: A deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1912–1920, 2015.
[5]
Ma, X.; Qin, C.; You, H. X.; Ran, H. X.; Fu, Y. Rethinking network design and local geometry in point cloud: A simple residual MLP framework. arXiv preprint arXiv:2202.07123, 2022.
Google Scholar
[6]
Thomas, H.; Qi, C. R.; Deschaud, J. E.; Marcotegui, B.; Goulette, F.; Guibas, L. KPConv: Flexible and deformable convolution for point clouds. In: Pro-ceedings of the IEEE/CVF International Conference on Computer Vision, 6410–6419, 2019.
Crossref
[7]
Liu, Y. C.; Fan, B.; Xiang, S. M.; Pan, C. H. Relation-shape convolutional neural network for point cloud analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8887–8896, 2019.
Crossref
[8]
Wang, Y. E.; Sun, Y. B.; Liu, Z. W.; Sarma, S. E.; Bronstein, M. M.; Solomon, J. M. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics Vol. 38, No. 5, Article No. 146, 2019.
Crossref Google Scholar
[9]
Rao, Y. M.; Lu, J. W.; Zhou, J. Global-local bidirectional reasoning for unsupervised representation learning of 3D point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5375–5384, 2020.
Crossref
[10]
Qin, C.; You, H. X.; Wang, L. C.; Kuo, C. C. J.; Fu, Y. PointDAN: A multi-scale 3D domain adaption network for point cloud representation. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, Article No. 646, 7192–7203, 2019.
[11]
Xu, M. Y.; Zhou, Z. P.; Qiao, Y. Geometry sharing network for 3D point cloud classification and segmentation. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 34, No. 7, 12500–12507, 2020.
Crossref Google Scholar
[12]
Charles, R. Q.; Hao, S.; Mo, K. C.; Guibas, L. J. PointNet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 77–85, 2017.
Crossref
[13]
Qi, C. R.; Yi, L.; Su, H.; Guibas, L. J. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, 5105–5114, 2017.
[14]
Xu, M. T.; Ding, R. Y.; Zhao, H. S.; Qi, X. J. PAConv: Position adaptive convolution with dynamic kernel assembling on point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3172–3181, 2021.
Crossref
[15]
Yang, S.; Xu, J.; Chen, K.; Fu, H. B. View suggestion for interactive segmentation of indoor scenes. Computational Visual Media Vol. 3, No. 2, 131–146, 2017.
Crossref Google Scholar
[16]
Li, Y.; Bu, R.; Sun, M.; Wu, W.; Di, X.; Chen, B. PointCNN: Convolution on X-transformed points. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, 828–838, 2018.
[17]
Wu, W. X.; Qi, Z. A.; Li, F. X. PointConv: Deep convolutional networks on 3D point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9613–9622, 2019.
Crossref
[18]
Yan, X.; Zheng, C. D.; Li, Z.; Wang, S.; Cui, S. G. PointASNL: Robust point clouds processing using nonlocal neural networks with adaptive sampling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5588–5597, 2020.
Crossref
[19]
Bengio, Y.; Courville, A.; Vincent, P. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 8, 1798–1828, 2013.
Crossref Google Scholar
[20]
Bachman, P.; Hjelm, R. D.; Buchwalter, W. Learning representations by maximizing mutual information across views. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, Article No. 1392, 15535–15545, 2019.
[21]
Doersch, C.; Gupta, A.; Efros, A. A. Unsupervised visual representation learning by context prediction. In: Proceedings of the IEEE International Conference on Computer Vision, 1422–1430, 2015.
Crossref
[22]
Doersch, C.; Zisserman, A. Multi-task self-supervised visual learning. In: Proceedings of the IEEE International Conference on Computer Vision, 2070–2079, 2017.
Crossref
[23]
Hénaff, O. J.; Srinivas, A.; De Fauw, J.; Razavi, A.; Doersch, C.; Ali Eslami, S. M.; Van Den Oord, A. Data-efficient image recognition with contrastive predictive coding. In: Proceedings of the 37th International Conference on Machine Learning, Article No. 391, 4182–4192, 2020.
[24]
Tian, Y. L.; Krishnan, D.; Isola, P. Contrastive multiview coding. In: Computer Vision – ECCV 2020. Lecture Notes in Computer Science, Vol. 12356. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 776–794, 2020.
Crossref
[25]
Yang, Y. Q.; Feng, C.; Shen, Y. R.; Tian, D. FoldingNet: Point cloud auto-encoder via deep grid deformation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 206–215, 2018.
Crossref
[26]
Zhao, Y. H.; Birdal, T.; Deng, H. W.; Tombari, F. 3D point capsule networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1009–1018, 2019.
Crossref
[27]
Zhang, M.; Kadam, P.; Liu, S.; Kuo, C. C. J. Unsupervised feedforward feature (UFF) learning for point cloud classification and segmentation. In: Proceedings of the IEEE International Conference on Visual Communications and Image Processing, 144–147, 2020.
Crossref
[28]
Hassani, K.; Haley, M. Unsupervised multi-task feature learning on point clouds. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 8159–8170, 2019.
Crossref
[29]
Xu, M. Y.; Wang, Y. L.; Zhou, Z. P.; Xu, H. B.; Qiao, Y. CP-net: Contour-perturbed reconstruction network for self-supervised point cloud learning. arXiv preprint arXiv:2201.08215, 2022.
Google Scholar
[30]
Uy, M. A.; Pham, Q. H.; Hua, B. S.; Nguyen, T.; Yeung, S. K. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 1588–1597, 2019.
Crossref
[31]
Yi, L.; Kim, V. G.; Ceylan, D.; Shen, I. C.; Yan, M. Y.; Su, H.; Lu, C. W.; Huang, Q. X.; Sheffer, A.; Guibas, L. A scalable active framework for region annotation in 3D shape collections. ACM Transactions on Graphics Vol. 35, No. 6, Article No. 210, 2016.
Crossref Google Scholar
[32]
Mo, K. C.; Zhu, S. L.; Chang, A. X.; Yi, L.; Tripathi, S.; Guibas, L. J.; Su, H. PartNet: A large-scale benchmark for fine-grained and hierarchical part-level 3D object understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 909–918, 2019.
Crossref
[33]
Pan, L.; Chen, X. Y.; Cai, Z. A.; Zhang, J. Z.; Zhao, H. Y.; Yi, S.; Liu, Z. W. Variational relational point completion network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8520–8529, 2021.
Crossref
[34]
Zhou, F.; Jiang, Z. Q.; Shui, C. J.; Wang, B. Y.; Chaib-draa, B. Domain generalization via optimal transport with metric similarity learning. arXiv preprint arXiv:2007.10573, 2020.
Crossref Google Scholar
[35]
Dai, A.; Chang, A. X.; Savva, M.; Halber, M.; Funkhouser, T.; Nießner, M. ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2432–2443, 2017.
Crossref
[36]
Xu, Y.; Fan, T.; Xu, M.; Zeng, L.; Qiao, Y. SpiderCNN: Deep learning on point sets with parameterized convolutional filters. In: Computer Vision – ECCV 2018. Lecture Notes in Computer Science, Vol. 11212. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 90–105, 2018.
Crossref
[37]
Guo, M. H.; Cai, J. X.; Liu, Z. N.; Mu, T. J.; Martin, R. R.; Hu, S. M. PCT: Point cloud transformer. Computational Visual Media Vol. 7, No. 2, 187–199, 2021.
Crossref Google Scholar
[38]
Xu, M. T.; Zhang, J. H.; Zhou, Z. P.; Xu, M. Y.; Qi, X. J.; Qiao, Y. Learning geometry-disentangled representation for complementary understanding of 3D object point cloud. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 35, No. 4, 3056–3064, 2021.
Crossref Google Scholar
[39]
Hua, B. S.; Tran, M. K.; Yeung, S. K. Pointwise convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 984–993, 2018.
Crossref
[40]
Simonovsky, M.; Komodakis, N. Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 29–38, 2017.
Crossref
[41]
Xie, S. N.; Liu, S. N.; Chen, Z. Y.; Tu, Z. W. Attentional ShapeContextNet for point cloud recognition. In: Proceedings of the IEEE/CVF Con-ference on Computer Vision and Pattern Recognition, 4606–4615, 2018.
Crossref
[42]
Groh, F.; Wieschollek, P.; Lensch, H. P. A. Flex-convolution (million-scale point-cloud learning beyond grid-worlds). arXiv preprint arXiv:1803.07289, 2018.
Google Scholar
[43]
Li, J. X.; Chen, B. M.; Lee, G. H. SO-net: Self-organizing network for point cloud analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9397–9406, 2018.
Crossref
[44]
Shen, Y. R.; Feng, C.; Yang, Y. Q.; Tian, D. Mining point cloud local structures by kernel correlation and graph pooling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4548–4557, 2018.
Crossref
[45]
Gadelha, M.; Wang, R.; Maji, S. Multiresolution tree networks for 3D point cloud processing. In: Computer Vision – ECCV 2018. Lecture Notes in Computer Science, Vol. 11211. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 105–122, 2018.
Crossref
[46]
Wang, C.; Samari, B.; Siddiqi, K. Local spectral graph convolution for point set feature learning. In: Computer Vision – ECCV 2018. Lecture Notes in Computer Science, Vol. 11208. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 56–71, 2018.
Crossref
[47]
Yang, J. C.; Zhang, Q.; Ni, B. B.; Li, L. G.; Liu, J. X.; Zhou, M. D.; Tian, Q. Modeling point clouds with self-attention and gumbel subset sampling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3318–3327, 2019.
Crossref
[48]
Klokov, R.; Lempitsky, V. Escape from cells: Deep kd-networks for the recognition of 3D point cloud models. In: Proceedings of the IEEE International Conference on Computer Vision, 863–872, 2017.
Crossref
[49]
Atzmon, M.; Maron, H.; Lipman, Y. Point convo-lutional neural networks by extension operators. arXiv preprint arXiv:1803.10091, 2018.
Crossref Google Scholar
[50]
Zhong, Q.; Han, X. F. Point cloud learning with transformer. arXiv preprint arXiv:2104.13636, 2021.
Crossref Google Scholar
[51]
Zhang, M.; You, H. X.; Kadam, P.; Liu, S.; Kuo, C. C J. PointHop: An explainable machine learning method for point cloud classification. IEEE Transactions on Multimedia Vol. 22, No. 7, 1744–1755, 2020.
Crossref Google Scholar
[52]
Kingma, D. P.; Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Google Scholar
[53]
Cortes, C.; Vapnik, V. Support-vector networks. Machine Learning Vol. 20, No. 3, 273–297, 1995.
Crossref Google Scholar
[54]
Ganin, Y.; Ustinova, E.; Ajakan, H.; Germain, P.; Larochelle, H.; Laviolette, F.; Marchand, M.; Lempitsky, V. Domain-adversarial training of neural networks. In: Domain Adaptation in Computer Vision Applications. Advances in Computer Vision and Pattern Recognition. Csurka, G. Ed. Springer Cham, 189–209, 2017.
Crossref
[55]
Sauder, J.; Sievers, B. Self-supervised deep learning on point clouds by reconstructing space. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, Article No. 1161, 12962–12972, 2019.
[56]
Achituve, I.; Maron, H.; Chechik, G. Self-supervised learning for domain adaptation on point clouds. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 123–133, 2021.
Crossref
[57]
Alliegro, A.; Boscaini, D.; Tommasi, T. Joint supervised and self-supervised learning for 3D real world challenges. In: Proceedings of the 25th International Conference on Pattern Recognition, 6718–6725, 2020.
[58]
Wang, S. L.; Suo, S.; Ma, W. C.; Pokrovsky, A.; Urtasun, R. Deep parametric continuous convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2589–2597, 2018.
Crossref
[59]
Komarichev, A.; Zhong, Z. C.; Hua, J. A-CNN: Annularly convolutional neural networks on point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7413–7422, 2019.
Crossref
[60]
Zhao, H. S.; Jiang, L.; Fu, C. W.; Jia, J. Y. PointWeb: Enhancing local neighborhood features for point cloud processing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5560–5568, 2019.
Crossref
[61]
Zhang, Z. Y.; Hua, B. S.; Yeung, S. K. ShellNet: Efficient point cloud convolutional neural networks using concentric shells statistics. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 1607–1616, 2019.
Crossref
[62]
Han, W. K.; Wen, C. L.; Wang, C.; Li, X.; Li, Q. Point2Node: Correlation learning of dynamic-node for point cloud feature modeling. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 34, No. 7, 10925–10932, 2020.
Crossref Google Scholar
[63]
Hu, Q. Y.; Yang, B.; Xie, L. H.; Rosa, S.; Guo, Y. L.; Wang, Z. H.; Trigoni, N.; Markham, A. RandLA-net: Efficient semantic segmentation of large-scale point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11105–11114, 2020.
Crossref
[64]
Lin, Y. Q.; Yan, Z. Z.; Huang, H. B.; Du, D.; Liu, L. G.; Cui, S. G.; Han, X. G. FPConv: Learning local flattening for point convolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4292–4301, 2020.
Crossref
[65]
Huang, Q. G.; Wang, W. Y.; Neumann, U. Recurrent slice networks for 3D segmentation of point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2626–2635, 2018.
Crossref
[66]
Chang, A. X.; Funkhouser, T.; Guibas, L.; Hanrahan, P.; Huang, Q. X.; Li, Z. M.; Savarese, S.; Savva, M.; Song, S. R.; Su, H.; et al. ShapeNet: An information-rich 3D model repository. arXiv preprint arXiv:1512.03012, 2015.
Google Scholar
[67]
Armeni, I.; Sener, O.; Zamir, A. R.; Jiang, H.; Brilakis, I.; Fischer, M.; Savarese, S. 3D semantic parsing of large-scale indoor spaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1534–1543, 2016.
Crossref
[68]
Boulch, A. ConvPoint: Continuous convolutions for point cloud processing. Computers & Graphics Vol. 88, 24–34, 2020.
Crossref Google Scholar
[69]
Xu, M. Y.; Zhou, Z. P.; Zhang, J. H.; Qiao, Y. Investigate indistinguishable points in semantic segmentation of 3D point cloud. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 35, No. 4, 3047–3055, 2021.
Crossref Google Scholar
[70]
Chen, Y. L.; Hu, V. T.; Gavves, E.; Mensink, T.; Mettes, P.; Yang, P. W.; Snoek, C. G. M. PointMixup: Augmentation for point clouds. In: Computer Vision – ECCV 2020. Lecture Notes in Computer Science, Vol. 12348. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 330–345, 2020.
Crossref
Computational Visual Media
Volume 10 Issue 1,
February 2024
Pages 27-43
DOI: 10.1007/s41095-022-0305-5
Cite this article:
Xu M, Zhou Z, Wang Y, et al. Towards robustness and generalization of point cloud representation: A geometry coding method and a large-scale object-level dataset. Computational Visual Media, 2024, 10(1): 27-43. https://doi.org/10.1007/s41095-022-0305-5

404

Views

10

Downloads

1

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 19 May 2022
Accepted: 21 July 2022
Published: 30 November 2023
© The Author(s) 2023.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
Return