RoomDesigner: Encoding Anchor-latents for Style-consistent and Shape-compatible Indoor Scene Generation

doi:10.1109/3DV62453.2024.00132

	RoomDesigner: Encoding Anchor-latents for Style-consistent and Shape-compatible Indoor Scene Generation
	Zhao, Yiqun1 ; Zhao, Zibo1 ; Li, Jing 2; Dong, Sixun1 ; Gao, Shenghua1
	2024
会议录名称	PROCEEDINGS - 2024 INTERNATIONAL CONFERENCE ON 3D VISION, 3DV 2024
ISSN	2378-3826
页码	1413-1423
发表状态	已发表
DOI	10.1109/3DV62453.2024.00132
摘要	Indoor scene generation aims at creating shape-compatible, style-consistent furniture arrangements within a spatially reasonable layout. However, most existing approaches primarily focus on generating plausible furniture layouts without incorporating specific details related to individual furniture. To address this limitation, we propose a two-stage model integrating shape priors into the indoor scene generation by encoding furniture as anchor latent representations. In the first stage, we employ discrete vector quantization to encode each piece of furniture as anchor-latents. Based on the anchor-latents representation, the shape and location information of furniture was characterized by a concatenation of location, size, orientation, class, and our anchor latent. In the second stage, we leverage a transformer model to predict indoor scenes configuration autoregressively. Thanks to the proposed anchor-latents representations, our generative model can synthesis furniture in diverse shapes and produce physically plausible arrangements with shape-compatible and style-consistent furniture. Furthermore, our method facilitates various human interaction applications, such as style-consistent scene completion, object mismatch correction, and controllable object-level editing. Experimental results on the 3D-Front dataset demonstrate that our approach can generate more consistent and compatible indoor scenes compared to existing methods, even without shape retrieval. Additionally, extensive ablation studies confirm the effectiveness of our design choices in the indoor scene generation model. © 2024 IEEE.
关键词	Computer vision Signal encoding 3d scene generation 3D scenes Discrete vector Encodings Location information Scene generation Shape information Shape priors Two stage model Vector quantisation
会议名称	11th International Conference on 3D Vision, 3DV 2024
出版地	10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA
会议地点	Davos, Switzerland
会议日期	March 18, 2024 - March 21, 2024
URL	查看原文
收录类别	EI ; CPCI-S
语种	英语
资助项目	NSFC["62172279","61932020"]
WOS研究方向	Computer Science ; Engineering ; Imaging Science & Photographic Technology
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic ; Imaging Science & Photographic Technology
WOS记录号	WOS:001250581700119
出版者	Institute of Electrical and Electronics Engineers Inc.
EI入藏号	20242616355256
EI主题词	Encoding (symbols)
EISSN	2475-7888
EI分类号	716.1 Information Theory and Signal Processing ; 723.2 Data Processing and Image Processing ; 723.5 Computer Applications ; 741.2 Vision
原始文献类型	Conference article (CA)
来源库	IEEE
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/395998
专题	信息科学与技术学院_硕士生信息科学与技术学院_PI研究组_高盛华组信息科学与技术学院_博士生
通讯作者	Zhao, Yiqun
作者单位	1.ShanghaiTech University, China 2.Xiaohongshu Inc, China
第一作者单位	上海科技大学
通讯作者单位	上海科技大学
第一作者的第一单位	上海科技大学
推荐引用方式 GB/T 7714	Zhao, Yiqun,Zhao, Zibo,Li, Jing,et al. RoomDesigner: Encoding Anchor-latents for Style-consistent and Shape-compatible Indoor Scene Generation[C]. 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA:Institute of Electrical and Electronics Engineers Inc.,2024:1413-1423.