Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Learning to reconstruct and understand indoor scenes from sparse views

Yang, Jingyu, Xu, Ji, Li, Kun, Lai, Yukun, Yue, Huanjing, Lu, Jianzhi, Wu, Hao and Liu, Yebin 2020. Learning to reconstruct and understand indoor scenes from sparse views. IEEE Transactions on Image Processing 29 , 5753 -5766. 10.1109/TIP.2020.2986712

[img]
Preview
PDF - Accepted Post-Print Version
Download (17MB) | Preview

Abstract

This paper proposes a new method for simultaneous 3D reconstruction and semantic segmentation for indoor scenes. Unlike existing methods that require recording a video using a color camera and/or a depth camera, our method only needs a small number of (e.g., 3~5) color images from uncalibrated sparse views, which significantly simplifies data acquisition and broadens applicable scenarios. To achieve promising 3D reconstruction from sparse views with limited overlap, our method first recovers the depth map and semantic information for each view, and then fuses the depth maps into a 3D scene. To this end, we design an iterative deep architecture, named IterNet, to estimate the depth map and semantic segmentation alternately. To obtain accurate alignment between views with limited overlap, we further propose a joint global and local registration method to reconstruct a 3D scene with semantic information. We also make available a new indoor synthetic dataset, containing photorealistic high-resolution RGB images, accurate depth maps and pixel-level semantic labels for thousands of complex layouts. Experimental results on public datasets and our dataset demonstrate that our method achieves more accurate depth estimation, smaller semantic segmentation errors, and better 3D reconstruction results over state-of-the-art methods.

Item Type: Article
Date Type: Publication
Status: Published
Schools: Computer Science & Informatics
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
ISSN: 1057-7149
Date of First Compliant Deposit: 6 April 2020
Date of Acceptance: 28 March 2020
Last Modified: 11 Jun 2020 18:44
URI: http://orca-mwe.cf.ac.uk/id/eprint/130860

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics