AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Computer Science and Technology Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Regular Paper

3D Object Tracking with Adaptively Weighted Local Bundles

School of Software, Shandong University, Jinan 250101, China
School of Computer Science and Technology, Shandong University, Qingdao 266237, China
College of Engineering and Computing, University of South Carolina, Columbia 29208, U.S.A.

Xue-Ying Qin and Fan Zhong both supervised this work and provided funding support

Show Author Information

Abstract

The 3D object tracking from a monocular RGB image is a challenging task. Although popular color and edge-based methods have been well studied, they are only applicable to certain cases and new solutions to the challenges in real environment must be developed. In this paper, we propose a robust 3D object tracking method with adaptively weighted local bundles called AWLB tracker to handle more complicated cases. Each bundle represents a local region containing a set of local features. To alleviate the negative effect of the features in low-confidence regions, the bundles are adaptively weighted using a spatially-variant weighting function based on the confidence values of the involved energy terms. Therefore, in each frame, the weights of the energy items in each bundle are adapted to different situations and different regions of the same frame. Experiments show that the proposed method can improve the overall accuracy in challenging cases. We then verify the effectiveness of the proposed confidence-based adaptive weighting method using ablation studies and show that the proposed method overperforms the existing single-feature methods and multi-feature methods without adaptive weighting.

Keywords

feature fusion3D trackinglocal bundleconfidence map

Electronic Supplementary Material

Download File(s)
jcst-36-3-555-Highlights.pdf (150.4 KB)

References

[1]

Lepetit V, Fua P. Monocular model-based 3D tracking of rigid objects: A survey. Found. Trends ® in Comput. Graph. Vis., 2005, 1(1): 1-89. DOI: 10.1561/0600000001
Crossref Google Scholar
[2]

Vacchetti L, Lepetit V, Fua P. Stable real-time 3D tracking using online and offline information. IEEE Trans. Pattern Anal. Mach. Intell., 2004, 26(10): 1385-1391. DOI: 10.1109/TPAMI.2004.92.
Crossref Google Scholar
[3]
Lourakis M I A, Zabulis X. Model-based pose estimation for rigid objects. In Proc. the 9th International Conference on Computer Vision Systems, July 2013, pp.83-92. DOI: 10.1007/978-3-642-39402-7_9.
Crossref
[4]
Tan D J, Tombari F, Ilic S, Navab N. A versatile learning-based 3D temporal tracker: Scalable, robust, online. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.693-701. DOI: 10.1109/ICCV.2015.86.
Crossref
[5]

Besl P J, McKay N D. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell., 1992, 14(2): 239-256. DOI: 10.1109/34.121791.
Crossref Google Scholar
[6]
Peng S, Liu Y, Huang Q, Zhou X, Bao H. PVNet: Pixel-wise voting network for 6DoF pose estimation. In Proc. the 2019 IEEE Conference on Computer Vision and Pattern Recognition, June 2019, pp.4561-4570. DOI: 10.1109/CVPR.2019.00469.
Crossref
[7]

Ye Y, Zhang C, Hao X. ARPNET: Attention region proposal network for 3D object detection. Sci. China Inf. Sci., 2019, 62(12): Article No. 220104. DOI: 10.1007/s11432-019-2636-x.
Crossref Google Scholar
[8]

Garon M, Lalonde J. Deep 6-DOF tracking. IEEE Trans. Vis. Comput. Graph., 2017, 23(11): 2410-2418. DOI: 10.1109/TVCG.2017.2734599.
Crossref Google Scholar
[9]

Li Y, Wang G, Ji X, Xiang Y, Fox D. DeepIM: Deep iterative matching for 6D pose estimation. Int. J. Comput. Vis., 2020, 128(3): 657-678. DOI: 10.1007/s11263-019-01250-9.
Crossref Google Scholar
[10]
Harris C, Stennett C. RAPID—A video rate object tracker. In Proc. the 1990 British Machine Vision Conference, September 1990, pp.73-77. DOI: 10.5244/C.4.15.
Crossref
[11]

Seo B, Park H, Park J, Hinterstoisser S, Ilic S. Optimal local searching for fast and robust textureless 3D object tracking in highly cluttered backgrounds. IEEE Trans. Vis. Comput. Graph., 2014, 20(1): 99-110. DOI: 10.1109/TVCG.2013.94.
Crossref Google Scholar
[12]

Wang G, Wang B, Zhong F, Qin X, Chen B. Global optimal searching for textureless 3D object tracking. The Visual Computer, 2015, 31(6/7/8): 979-988. DOI: 10.1007/s00371-015-1098-7.
Crossref Google Scholar
[13]

Wang B, Zhong F, Qin X. Robust edge-based 3D object tracking with direction-based pose validation. Multimedia Tools Appl., 2019, 78(9): 12307-12331. DOI: 10.1007/s11042-018-6727-5.
Crossref Google Scholar
[14]

Zhang Y, Li X, Liu H, Shang Y. Comparative study of visual tracking method: A probabilistic approach for pose estimation using lines. IEEE Trans. Circuits Syst. Video Technol., 2017, 27(6): 1222-1234. DOI: 10.1109/TCSVT.2016.2527219.
Crossref Google Scholar
[15]

Prisacariu V A, Reid I D. PWP3D: Real-time segmentation and tracking of 3D objects. Int. J. Comput. Vis., 2012, 98(3): 335-354. DOI: 10.1007/s11263-011-0514-3.
Crossref Google Scholar
[16]
Tjaden H, Schwanecke U, Schömer E. Real-time monocular segmentation and pose tracking of multiple objects. In Proc. the 14th European Conference on Computer Vision, October 2016, pp.423-438. DOI: 10.1007/978-3-319-46493-0_26.
Crossref
[17]

Hexner J, Hagege R R. 2D-3D pose estimation of heterogeneous objects using a region based approach. Int. J. Comput. Vis., 2016, 118(1): 95-112. DOI: 10.1007/s11263-015-0873-2.
Crossref Google Scholar
[18]
Tjaden H, Schwanecke U, Schömer E. Real-time monocular pose estimation of 3D objects using temporally consistent local color histograms. In Proc. the 2017 IEEE International Conference on Computer Vision, October 2017, pp.124-132. DOI: 10.1109/ICCV.2017.23.
Crossref
[19]

Tjaden H, Schwanecke U, Schömer E, Cremers D. A region-based gauss-newton approach to real-time monocular multiple object tracking. IEEE Trans. Pattern Anal. Mach. Intell., 2019, 41(8): 1797-1812. DOI: 10.1109/TPAMI.2018.2884990.
Crossref Google Scholar
[20]

Marchand É, Bouthemy P, Chaumette F. A 2D-3D model-based approach to real-time visual tracking. Image Vis. Comput., 2001, 19(13): 941-955. DOI: 10.1016/S0262-8856(01)00054-3.
Crossref Google Scholar
[21]

Drummond T, Cipolla R. Real-time visual tracking of complex structures. IEEE Trans. Pattern Anal. Mach. Intell., 2002, 24(7): 932-946. DOI: 10.1109/TPAMI.2002.1017620.
Crossref Google Scholar
[22]
Wuest H, Vial F, Stricker D. Adaptive line tracking with multiple hypotheses for augmented reality. In Proc. the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality, October 2005, pp.62-69. DOI: 10.1109/ISMAR.2005.8.
Crossref
[23]

Choi C, Christensen H I. Robust 3D visual tracking using particle filtering on the special Euclidean group: A combined approach of keypoint and edge features. The International Journal of Robotics Research, 2012, 31(4): 498-519. DOI: 10.1177/0278364912437213.
Crossref Google Scholar
[24]
Wang B, Zhong F, Qin X. Pose optimization in edge distance field for textureless 3D object tracking. In Proc. the 2017 Computer Graphics International Conference, June 2017, Article No. 32. DOI: 10.1145/3095140.3095172.
Crossref
[25]

Osher S, Sethian J A. Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics, 1988, 79(1): 12-49. DOI: 10.1016/0021-9991(88)90002-2.
Crossref Google Scholar
[26]

Zhong L, Zhao X, Zhang Y, Zhang S, Zhang L. Occlusion-aware region-based 3D pose tracking of objects with temporally consistent polar-based local partitioning. IEEE Trans. Image Process., 2020, 29: 5065-5078. DOI: 10.1109/TIP.2020.2973512.
Crossref Google Scholar
[27]

Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278-2324. DOI: 10.1109/5.726791.
Crossref Google Scholar
[28]

Crivellaro A, Rad M, Verdie Y, Yi K M, Fua P, Lepetit V. Robust 3D object tracking from monocular images using stable parts. IEEE Trans. Pattern Anal. Mach. Intell., 2018, 40(6): 1465-1479. DOI: 10.1109/TPAMI.2017.2708711.
Crossref Google Scholar
[29]

Zhong L, Zhang L. A robust monocular 3D object tracking method combining statistical and photometric constraints. Int. J. Comput. Vis., 2019, 127(8): 973-992. DOI: 10.1007/s11263-018-1119-x.
Crossref Google Scholar
[30]
Ma Y, Soatto S, Košecká J, Sastry S S. An Invitation to 3-D Vision: From Images to Geometric Models (1st edition). Springer-Verlag New York Publishers, 2004.
[31]
Zhong F, Qin X, Chen J, Hua W, Peng Q. Confidence-based color modeling for online video segmentation. In Proc. the 9th Asian Conference on Computer Vision, September 2009, pp.697-706. DOI: 10.1007/978-3-642-12304-7_66.
Crossref
[32]
Wu P, Lee Y, Tseng H, Ho H, Yang M, Chien S. A benchmark dataset for 6DoF object pose tracking. In Proc. the 2017 IEEE International Symposium on Mixed and Augmented Reality Adjunct, October 2017, pp.186-191. DOI: 10.1109/ISMAR-Adjunct.2017.62.
Crossref
[33]
Brachmann E, Michel F, Krull A, Yang M Y, Gumhold S, Rother C. Uncertainty-driven 6D pose estimation of objects and scenes from a single RGB image. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.3364-3372. DOI: 10.1109/CVPR.2016.366.
Crossref
[34]
Whelan T, Leutenegger S, Salas-Moreno R F, Glocker B, Davison A J. ElasticFusion: Dense SLAM without a pose graph. In Proc. the 2015 Robotics: Science and Systems, July 2015. DOI: 10.15607/RSS.2015.XI.001.
Crossref
[35]

Mur-Artal R, Tardós J D. ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Trans. Robotics, 2017, 33(5): 1255-1262. DOI: 10.1109/TRO.2017.2705103.
Crossref Google Scholar
[36]

Marchand É, Uchiyama H, Spindler F. Pose estimation for augmented reality: A hands-on survey. IEEE Trans. Vis. Comput. Graph., 2016, 22(12): 2633-2651. DOI: 10.1109/TVCG.2015.2513408.
Crossref Google Scholar
[37]

Cheng M, Liu Y, Lin W, Zhang Z, Rosin P L, Torr P H S. BING: Binarized normed gradients for objectness estimation at 300fps. Comput. Vis. Media, 2019, 5(1): 3-20. DOI: 10.1007/s41095-018-0120-1.
Crossref Google Scholar
Journal of Computer Science and Technology
Volume 36 Issue 3,
May 2021
Pages 555-571
DOI: 10.1007/s11390-021-1272-5
Cite this article:
Li J-C, Zhong F, Xu S-H, et al. 3D Object Tracking with Adaptively Weighted Local Bundles. Journal of Computer Science and Technology, 2021, 36(3): 555-571. https://doi.org/10.1007/s11390-021-1272-5

408

Views

13

Crossref

11

Web of Science

13

Scopus

2

CSCD

Altmetrics
Received: 06 January 2021
Accepted: 23 April 2021
Published: 05 May 2021
©Institute of Computing Technology, Chinese Academy of Sciences 2021
Return