AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Computational Visual Media Article
PDF (19.9 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

3D floor plan recovery from overlapping spherical images

Giovanni Pintore1( ), Fabio Ganovelli1, Ruggero Pintus1, Roberto Scopigno1, Enrico Gobbetti2
CRS4, Visual Computing Group, Cagliari, Italy.
CNR-ISTI, Visual Computing Group, Pisa, Italy.
Show Author Information

Abstract

We present a novel approach to automati-cally recover, from a small set of partially overlapping spherical images, an indoor structure representation in terms of a 3D floor plan registered with a set of 3D environment maps. We introduce several improvements over previous approaches based on color and spatial reasoning exploiting Manhattan world priors. Inparticular, we introduce a new method for geometric context extraction based on a 3D facet representation, which combines color distribution analysis of individual images with sparse multi-view clues. We also introduce an efficient method to combine the facets from different viewpoints in a single consistent model, taking into the reliability of the facet information. The resulting capture and reconstruction pipeline automatically generates 3D multi-room environments in cases where most previous approaches fail, e.g., in the presence of hidden corners and large clutter, without the need for additional dense 3D data or tools. We demonstrate the effectiveness and performance of our approach on different real-world indoor scenes. Our test data is available to allow further studies and comparisons.

Keywords

indoor reconstructionspherical pano-ramic cameras360 degree photographymulti-room environments

References

[1]
J. Kopf, 360 video stabilization. ACM Transactions on Graphics Vol. 35, No. 6, Article No. 195, 2016.
Crossref Google Scholar
[2]
K. Matzen,; M. F. Cohen,; B. Evans,; J. Kopf,; R. Szeliski, Low-cost 360 stereo photography and video capture. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 148, 2017.
Crossref Google Scholar
[3]
M. Brown,; D. G. Lowe, Automatic panoramic image stitching using invariant features. International Journal of Computer Vision Vol. 74, No. 1, 59-73, 2007.
Crossref Google Scholar
[4]
G. Pintore,; V. Garro,; F. Ganovelli,; E. Gobbetti,; M. Agus, Omnidirectional image capture on mobile devices for fast automatic generation of 2.5D indoor maps. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 1-9, 2016.
Crossref
[5]
G. Pintore,; E. Gobbetti, Effective mobile mapping of multi-room indoor structures. The Visual Computer Vol. 30, Nos. 6-8, 707-716, 2014.
Crossref Google Scholar
[6]
G. Pintore,; F. Ganovelli,; E. Gobbetti,; R. Scopigno, Mobile mapping and visualization of indoor structures to simplify scene understanding and location awareness. In: Computer Vision—ECCV 2016 Workshops. Lecture Notes in Computer Science, Vol. 9914. G. Hua,; H. Jégou, Eds. Springer Cham, 130-145, 2016.
Crossref
[7]
H. Yang,; H. Zhang, Efficient 3D room shape recovery from a single panorama. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5422-5430, 2016.
Crossref
[8]
R. Cabral,; Y. Furukawa, Piecewise planar and compact floorplan reconstruction from images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 628-635, 2014.
Crossref
[9]
G. Pintore,; F. Ganovelli,; R. Pintus,; R. Scopigno,; E. Gobbetti, Recovering 3D indoor floor plans by exploiting low-cost spherical photography. In: Proceedings of the Pacific Graphics, 2018. Available at http://publications.crs4.it/pubdocs/2018/PGPSG18/pg2018s-indoorplan.pdf.
[10]
X. Xiong,; A. Adan,; B. Akinci,; D. Huber, Automatic creation of semantically rich 3D building models from laser scanner data. Automation in Construction Vol. 31, 325-337, 2013.
Crossref Google Scholar
[11]
C. Mura,; O. Mattausch,; A. J. Villanueva,; E. Gobbetti,; R. Pajarola, Automatic room detection and reconstruction in cluttered indoor environments with complex room layouts. Computers & Graphics Vol. 44, 20-32, 2014.
Crossref Google Scholar
[12]
C. Mura,; O. Mattausch,; R. Pajarola, Piecewise-planar reconstruction of multi-room interiors with arbitrary wall arrangements. Computer Graphics Forum Vol. 35, No. 7, 179-188, 2016.
Crossref Google Scholar
[13]
R. Guo,; D. Hoiem, Support surface prediction in indoor scenes. In: Proceedings of the IEEE International Conference on Computer Vision, 2144-2151, 2013.
Crossref
[14]
Z. Jia,; A. Gallagher,; A. Saxena,; T. Chen, 3D-based reasoning with blocks, support, and stability. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1-8, 2013.
Crossref
[15]
Google. Tango. 2014. Available at www.google.com/atap/projecttango/.
[16]
S. Ikehata,; H. Yang,; Y. Furukawa, Structured indoor modeling. In: Proceedings of the IEEE International Conference on Computer Vision, 1323-1331, 2015.
Crossref
[17]
Y. M. Kim,; N. J. Mitra,; D.-M. Yan,; L. Guibas, Acquiring 3D indoor environments with variability and repetition. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 138, 2012.
Crossref Google Scholar
[18]
L. Nan,; K. Xie,; A. Sharf, A search-classify approach for cluttered indoor scene understanding. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 137, 2012.
Crossref Google Scholar
[19]
Autodesk. 123D Catch. Available at www.123dapp.com/catch.
[20]
Microsoft. Photosynth. Available at photosynth.net/.
[21]
S. M. Seitz,; B. Curless,; J. Diebel,; D. Scharstein,; R. Szeliski, A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 519-528, 2006.
[22]
Y. Furukawa,; B. Curless,; S. M. Seitz,; R. Szeliski, Reconstructing building interiors from images. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 80-87, 2009.
Crossref
[23]
A. Flint,; D. Murray,; I. Reid, Manhattan scene understanding using monocular, stereo, and 3D features. In: Proceedings of the International Conference on Computer Vision, 2228-2235, 2011.
Crossref
[24]
G. Tsai,; C. Xu,; J. Liu,; B. Kuipers, Real-time indoor scene understanding using Bayesian filtering with motion cues. In: Proceedings of the International Conference on Computer Vision, 121-128, 2011.
Crossref
[25]
J. M. Coughlan,; A. L. Yuille, Manhattan world: Compass direction from a single image by Bayesian inference. In: Proceedings of the 7th IEEE International Conference on Computer Vision, Vol. 2, 941-947, 1999.
Crossref
[26]
S. Y. Bao,; A. Furlan,; L. Fei-Fei,; S. Savarese, Understanding the 3D layout of a cluttered room from multiple images. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 690-697, 2014.
Crossref
[27]
C. Häne,; L. Heng,; G. H. Lee,; A. Sizov,; M. Pollefeys, Real-time direct dense matching on fisheye images using plane-sweeping stereo. In: Proceedings of the 2nd International Conference on 3D Vision, 57-64, 2014.
Crossref
[28]
P. Chang,; M. Hebert, Omni-directional structure from motion. In: Proceedings of the IEEE Workshop on Omnidirectional Vision, 127-133, 2000.
[29]
M. Schönbein,; A. Geiger, Omnidirectional 3D reconstruction in augmented Manhattan worlds. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 716-723, 2014.
Crossref
[30]
B. Micusik,; T. Pajdla, Structure from motion with wide circular field of view cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 28, No. 7, 1135-1149, 2006.
Crossref Google Scholar
[31]
B. Micusik,; T. Pajdla, Autocalibration & 3D reconstruction with non-central catadioptric cameras. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, I-58-I-65, 2004.
[32]
R. Bunschoten,; B. Krose, Robust scene reconstruc-tion from an omnidirectional vision system. IEEE Transactions on Robotics and Automation Vol. 19, No. 2, 351-357, 2003.
Crossref Google Scholar
[33]
S. Zingg,; D. Scaramuzza,; S. Weiss,; R. Siegwart, MAV navigation through indoor corridors using optical flow. In: Proceedings of the IEEE International Conference on Robotics and Automation, 3361-3368, 2010.
Crossref
[34]
S. Li, Binocular spherical stereo. IEEE Transactions on Intelligent Transportation Systems Vol. 9, No. 4, 589-600, 2008.
Crossref Google Scholar
[35]
C. Geyer,; K. Daniilidis, A unifying theory for central panoramic systems and practical implications. In: Computer Vision—ECCV 2000. Lecture Notes in Computer Science, Vol. 1843. D. Vernon, Ed. Springer Berlin Heidelberg, 445-461, 2000.
Crossref
[36]
H. Kim,; A. Hilton, 3D scene reconstruction from multiple spherical stereo pairs. International Journal of Computer Vision Vol. 104, No. 1, 94-116, 2013.
Crossref Google Scholar
[37]
S. Im,; H. Ha,; F. Rameau,; H.-G. Jeon,; G. Choe,; I. S. Kweon, All-around depth from small motion with a spherical panoramic camera. In: Computer Vision—ECCV 2016. Lecture Notes in Computer Science, Vol. 9907. B. Leibe,; J. Matas,; N. Sebe,; M. Welling, Eds. Springer Cham, 156-172, 2016.
Crossref
[38]
D. Caruso,; J. Engel,; D. Cremers, Large-scale direct SLAM for omnidirectional cameras. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 141-148, 2015.
Crossref
[39]
G. Pintore,; R. Pintus,; F. Ganovelli,; R. Scopigno,; E. Gobbetti, Recovering 3D existing-conditions of indoor structures from spherical images. Computers & Graphics Vol. 77, 16-29, 2018.
Crossref Google Scholar
[40]
F. Kangni,; R. Laganiere, Orientation and pose recovery from spherical panoramas. In: Proceedings of the IEEE 11th International Conference on Computer Vision, 1-8, 2007.
Crossref
[41]
R. Achanta,; A. Shaji,; K. Smith,; A. Lucchi,; P. Fua,; S. Süsstrunk, SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 11, 2274-2282, 2012.
Crossref Google Scholar
[42]
R. Marroquim,; M. Kraus,; P. R. Cavalcanti, Efficient image reconstruction for point-based and line-based rendering. Computers & Graphics Vol. 32, No. 2, 189-203, 2008.
Crossref Google Scholar
[43]
R. Grompone von Gioi,; J. Jakubowicz,; J.-M. Morel,; G. Randall, LSD: A line segment detector. Image Processing On Line No. 2, 35-55, 2012.
Crossref Google Scholar
[44]
D. H. Douglas,; T. K. Peucker, Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: The International Journal for Geographic Information and Geovisualization Vol. 10, No. 2, 112-122, 1973.
Crossref Google Scholar
[45]
D. C. Lee,; M. Hebert,; T. Kanade, Geometric reasoning for single image structure recovery. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2136-2143, 2009.
Crossref
[46]
Y. Zhang,; S. Song,; P. Tan,; J. Xiao, PanoContext: A whole-room 3D context model for panoramic scene understanding. In: Computer Vision-ECCV 2014. Lecture Notes in Computer Science, Vol. 8694. D. Fleet,; T. Pajdla,; B. Schiele,; T. Tuytelaars, Eds. Springer Cham, 668-686, 2014.
[47]
G. Schindler,; F. Dellaert, Atlanta world: An expectation maximization framework for simultaneous low-level edge grouping and camera calibration in complex man-made environments. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, I-203-I-209, 2004.
[48]
A. G. Schwing,; R. Urtasun, Efficient exact inference for 3D indoor scene understanding. In: Computer Vision-ECCV 2012. Lecture Notes in Computer Science, Vol. 7577. A. Fitzgibbon,; S. Lazebnik,; P. Perona,; Y. Sato,; C. Schmid, Eds. Springer Berlin Heidelberg, 299-313, 2012.
Computational Visual Media
Volume 4 Issue 4,
December 2018
Pages 367-383
DOI: 10.1007/s41095-018-0125-9
Cite this article:
Pintore G, Ganovelli F, Pintus R, et al. 3D floor plan recovery from overlapping spherical images. Computational Visual Media, 2018, 4(4): 367-383. https://doi.org/10.1007/s41095-018-0125-9

801

Views

24

Downloads

21

Crossref

N/A

Web of Science

21

Scopus

0

CSCD

Altmetrics
Revised: 26 September 2018
Accepted: 14 October 2018
Published: 19 November 2018
© The Author(s) 2018

This article is published with open access at Springerlink.com

The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
Return