PDF (96 MB)
Collect
Submit Manuscript
Research Article | Open Access

View suggestion for interactive segmentation of indoor scenes

Tsinghua University, Beijing, China.
Massachusetts Institute of Technology, Cambridge, USA.
City University of Hong Kong, Hong Kong, China.
Show Author Information

Abstract

Point cloud segmentation is a fundamental problem. Due to the complexity of real-world scenes and the limitations of 3D scanners, interactive segmentation is currently the only way to cope with all kinds of point clouds. However, interactively segmenting complex and large-scale scenes is very time-consuming. In this paper, we present a novel interactive system for segmenting point cloud scenes. Our system automatically suggests a series of camera views, in which users can conveniently specify segmentation guidance. In this way, users may focus on specifying segmentation hints instead of manually searching for desirable views of unsegmented objects, thus significantly reducing user effort. To achieve this, we introduce a novel view preference model, which is based on a set of dedicated view attributes, with weights learned from a user study. We also introduce support relations for both graph-cut-based segmentation and finding similar objects. Our experiments show that our segmentation technique helps users quickly segment various types of scenes, outperforming alternative methods.

Electronic Supplementary Material

Video
41095_2017_78_MOESM1_ESM.mp4
Download File(s)
41095_2017_78_MOESM2_ESM.pdf (2.4 MB)

References

[1]
K. Lai,; L. Bo,; X. Ren,; D. Fox, Detection-based object labeling in 3D scenes. In: Proceedings of the IEEE International Conference on Robotics and Automation, 1330-1337, 2012.
[2]
A. E. Johnson,; M. Hebert, Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 21, No. 5, 433-449, 1999.
[3]
B. Zheng,; Y. Zhao,; J. C. Yu,; K. Ikeuchi,; S. C. Zhu, Beyond point clouds: Scene understanding by reasoning geometry and physics. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3127-3134, 2013.
[4]
D. Holz,; S. Behnke, Fast range image segmentation and smoothing using approximate surface reconstruction and region growing. In: Intelligent Autonomous Systems 12. S. Lee,; H. Cho,; K.-J. Yoon,; J. Lee, Eds. Springer Berlin Heidelberg, 61-73, 2013.
[5]
T. Rabbani,; F. A. van den Heuvel,; G. Vosselmann, Segmentation of point clouds using smoothness constraint. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences Vol. 36, No. 5, 248-253, 2006.
[6]
Y. Boykov,; G. Funka-Lea, Graph cuts and efficient N-D image segmentation. International Journal of Computer Vision Vol. 70, No. 2, 109-131, 2006.
[7]
A. Golovinskiy,; T. Funkhouser, Min-cut based segmentation of point clouds. In: Proceedings of the IEEE 12th International Conference on Computer Vision Workshops, 39-46, 2009.
[8]
D. Sedlacek,; J. Zara, Graph cut based point-cloud segmentation for polygonal reconstruction. In: Advances in Visual Computing. G. Bebis,; R. Boyle,; B. Parvin,; D. Koracin, et al. Eds. Springer Berlin Heidelberg, 218-227, 2009.
[9]
Y. M. Kim,; N. J. Mitra,; D.-M. Yan,; L. Guibas, Acquiring 3D indoor environments with variability and repetition. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 138, 2012.
[10]
L. Nan,; K. Xie,; A. Sharf, A search-classify approach for cluttered indoor scene understanding. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 137, 2012.
[11]
N. Silberman,; D. Hoiem,; P. Kohli,; R. Fergus, Indoor segmentation and support inference from RGBD images. In: Computer Vision–ECCV 2012. A. Fitzgibbon,; S. Lazebnik,; P. Perona,; Y. Sato,; C. Schmid, Eds. Springer Berlin Heidelberg, 746-760, 2012.
[12]
A. Nguyen,; B. Le, 3D point cloud segmentation: A survey. In: Proceedings of the 6th IEEE Conference on Robotics, Automation and Mechatronics, 225-230, 2013.
[13]
T. Shao,; W. Xu,; K. Zhou,; J. Wang,; D. Li,; B. Guo, An interactive approach to semantic modeling of indoor scenes with an RGBD camera. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 136, 2012.
[14]
X. Yuan,; H. Xu,; M. X. Nguyen,; A. Shesh,; B. Chen, Sketch-based segmentation of scanned outdoor environment models. In: Proceedings of the Eurographics Workshop on Sketch-Based Interfaces and Modeling, 19-26, 2005.
[15]
N. Silberman,; R. Fergus, Indoor scene segmentation using a structured light sensor. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 601-608, 2011.
[16]
J. Xiao,; A. Owens,; A. Torralba, SUN3D: A database of big spaces reconstructed using SfM and object labels. In: Proceedings of the IEEE International Conference on Computer Vision, 1625-1632, 2013.
[17]
A. X. Chang,; T. Funkhouser,; L. Guibas,; P. Hanrahan,; Q. Huang,; Z. Li,; S. Savarese,; M. Savva,; S. Song,; H. Su,; J. Xiao,; L. Yi,; F. Yu, ShapeNet: An information-rich 3D model repository. arXiv preprint arXiv:1512.03012, 2015.
[18]
S. Hinterstoisser,; V. Lepetit,; S. Ilic,; S. Holzer,; G. R. Bradski,; K. Konolige,; N. Navab, Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes. In: Computer Vision–ACCV 2012. K. M. Lee,; Y. Matsushita,; J. M. Rehg,; Z. Hu, Eds. Springer Berlin Heidelberg, 548-562, 2012.
[19]
K. Chen,; Y.-K. Lai,; Y.-X. Wu,; R. Martin,; S.-M. Hu, Automatic semantic modeling of indoor scenes from low-quality RGB-D data using contextual information. ACM Transactions on Graphics Vol. 33, No. 6, Article No. 208, 2014.
[20]
N. Silberman,; D. Sontag,; R. Fergus, Instance segmentation of indoor scenes using a coverage loss. In: Computer Vision–ECCV 2014. D. Fleet,; T. Pajdla,; B. Schiele,; T. Tuytelaars, Eds. Springer International Publishing, 616-631, 2014.
[21]
K. Chen,; Y. K. Lai,; S.-M. Hu, 3D indoor scene modeling from RGB-D data: a survey. Computational Visual Media Vol. 1, No. 4, 267-278, 2015.
[22]
C.-H. Shen,; S.-S. Huang,; H. Fu,; S.-M. Hu, Adaptive partitioning of urban facades. ACM Transactions on Graphics Vol. 30, No. 6, Article No. 184, 2011.
[23]
H. Zhang,; K. Xu,; W. Jiang,; J. Lin,; D. Cohen-Or,; B. Chen, Layered analysis of irregular facades via symmetry maximization. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 121, 2013.
[24]
O. Mattausch,; D. Panozzo,; C. Mura,; O. Sorkine-Hornung,; R. Pajarola, Object detection and classification from large-scale cluttered indoor scans. Computer Graphics Forum Vol. 33, No. 2, 11-21, 2014.
[25]
J. Valentin,; V. Vineet,; M.-M. Cheng,; D. Kim,; J. Shotton,; P. Kohli,; M. Nießner,; A. Criminisi,; S. Izadi,; P. Torr, SemanticPaint: Interactive 3D labeling and learning at your fingertips. ACM Transactions on Graphics Vol. 34, No. 5, Article No. 154, 2015.
[26]
Y.-S. Wong,; H.-K. Chu,; N. J. Mitra, SmartAnnotator an interactive tool for annotating indoor RGBD images. Computer Graphics Forum Vol. 34, No. 2, 447-457, 2015.
[27]
M. Christie,; P. Olivier, Camera control in computer graphics: Models, techniques and applications. In: Proceedings of the ACM SIGGRAPH ASIA 2009 Courses, Article No. 3, 2009.
[28]
W. R. Scott,; G. Roth,; J.-F. Rivest, View planning for automated three-dimensional object reconstruction and inspection. ACM Computing Surveys Vol. 35, No. 1, 64-96, 2003.
[29]
A. Secord,; J. Lu,; A. Finkelstein,; M. Singh,; A. Nealen, Perceptual models of viewpoint preference. ACM Transactions on Graphics Vol. 30, No. 5, Article No. 109, 2011.
[30]
P.-P. Vázquez,; M. Feixas,; M. Sbert,; W. Heidrich, Viewpoint selection using viewpoint entropy. In: Proceedings of the Vision Modeling and Visualization Conference, 273-280, 2001.
[31]
C. Andújar,; P. Vázquez,; M. Fairén, Way-Finder: Guided tours through complex walkthrough models. Computer Graphics Forum Vol. 23, No. 3, 499-508, 2004.
[32]
T.-Y. Li,; J.-M. Lien,; S.-Y. Chiu,; T.-H. Yu, Automatically generating virtual guided tours. In: Proceedings of the Computer Animation, 99-106, 1999.
[33]
M. Christie,; E. Languénou, A constraint-based approach to camera path planning. In: Smart Graphics. A. Butz,; A. Krüger,; P. Olivier, Eds. Springer Berlin Heidelberg, 172-181, 2003.
[34]
B. Salomon,; M. Garber,; M. C. Lin,; D. Manocha, Interactive navigation in complex environments using path planning. In: Proceedings of the Symposium on Interactive 3D Graphics, 41-50, 2003.
[35]
S. Choi,; Q.-Y. Zhou,; V. Koltun, Robust reconstruction of indoor scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5556-5565, 2015.
[36]
R. A. Newcombe,; S. Izadi,; O. Hilliges,; D. Molyneaux,; D. Kim,; A. J. Davison,; P. Kohli,; J. Shotton,; S. Hodges,; A. Fitzgibbon, KinectFusion: Real-time dense surface mapping and tracking. In: Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality, 127-136, 2011.
[37]
S. Ikehata,; H. Yang,; Y. Furukawa, Structured indoor modeling. In: Proceedings of the IEEE International Conference on Computer Vision, 1323-1331, 2015.
[38]
Y. Furukawa,; B. Curless,; S. M. Seitz,; R. Szeliski, Manhattan-world stereo. In: Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, 1422-1429, 2009.
[39]
V. Asha,; N. U. Bhajantri,; P. Nagabhushan, GLCM-based chi-square histogram distance for automatic detection of defects on patterned textures. International Journal of Computational Vision and Robotics Vol. 2, No. 4, 302-313, 2011.
[40]
C. Früh,; A. Zakhor, Constructing 3D city models by merging aerial and ground views. IEEE Computer Graphics and Applications Vol. 23, No. 6, 52-61, 2003.
[41]
M. Fisher,; M. Savva,; Y. Li,; P. Hanrahan,; M. Nießner, Activity-centric scene synthesis for functional 3D scene modeling. ACM Transactions on Graphics Vol. 34, No. 6, Article No. 179, 2015.
[42]
Z. C. Marton,; R. B. Rusu,; M. Beet, On fast surface reconstruction methods for large and noisy point clouds. In: Proceedings of the IEEE International Conference on Robotics and Automation, 3218-3223, 2009.
[43]
R. Tibshirani, Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Vol. 58, No. 1, 267-288, 1996.
[44]
Y. Boykov,; O. Veksler,; R. Zabih, Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 23, No. 11, 1222-1239, 2001.
[45]
A. Handa,; T. Whelan,; J. McDonald,; A. J. Davison, A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM. In: Proceedings of the IEEE International Conference on Robotics and Automation, 1524-1531, 2014.
[46]
W. S. Gosset, The probable error of a mean. Biometrika Vol. 6, No. 1, 1-25, 1908.
Computational Visual Media
Pages 131-146
Cite this article:
Yang S, Xu J, Chen K, et al. View suggestion for interactive segmentation of indoor scenes. Computational Visual Media, 2017, 3(2): 131-146. https://doi.org/10.1007/s41095-017-0078-4
Metrics & Citations  
Article History
Copyright
Rights and Permissions
Return