• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
成果搜索

author:

Jiang, Yutao (Jiang, Yutao.) [1] | Zhou, Yang (Zhou, Yang.) [2] | Liang, Yuan (Liang, Yuan.) [3] | Liu, Wenxi (Liu, Wenxi.) [4] (Scholars:刘文犀) | Jiao, Jianbo (Jiao, Jianbo.) [5] | Quan, Yuhui (Quan, Yuhui.) [6] | He, Shengfeng (He, Shengfeng.) [7]

Indexed by:

CPCI-S EI

Abstract:

This paper aims to resolve the challenging problem of wide-angle novel view synthesis from a single image, a.k.a. wide-angle 3D photography. Existing approaches rely on local context and treat them equally to inpaint occluded RGB and depth regions, which fail to deal with large-region occlusion (i.e., observing from an extreme angle) and foreground layers might blend into background inpainting. To address the above issues, we propose Diffuse3D which employs a pre-trained diffusion model for global synthesis, while amending the model to activate depth-aware inference. Our key insight is to alter the convolution mechanism in the denoising process. We inject depth information into the denoising convolution operation with bilateral kernels, i.e., a depth kernel and a spatial kernel, to consider layered correlations among pixels. In this way, foreground regions are overlooked in background inpainting and only pixels close in depth are leveraged. On the other hand, we propose a global-local balancing approach to maximize both contextual understandings. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods in novel view synthesis, especially in wide-angle scenarios. More importantly, our method does not require any training and is a plug-and-play module that can be integrated with any diffusion model. Our code can be found at https://github.com/yutaojiang1/Diffuse3D.

Keyword:

Community:

  • [ 1 ] [Jiang, Yutao]South China Univ Technol, Sch Comp Sci & Engn, Guangzhou, Peoples R China
  • [ 2 ] [Zhou, Yang]South China Univ Technol, Sch Comp Sci & Engn, Guangzhou, Peoples R China
  • [ 3 ] [Liang, Yuan]South China Univ Technol, Sch Comp Sci & Engn, Guangzhou, Peoples R China
  • [ 4 ] [Quan, Yuhui]South China Univ Technol, Sch Comp Sci & Engn, Guangzhou, Peoples R China
  • [ 5 ] [Liu, Wenxi]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China
  • [ 6 ] [Jiao, Jianbo]Univ Birmingham, Sch Comp Sci, Birmingham, W Midlands, England
  • [ 7 ] [Jiang, Yutao]Singapore Management Univ, Sch Comp & Informat Syst, Singapore, Singapore
  • [ 8 ] [Zhou, Yang]Singapore Management Univ, Sch Comp & Informat Syst, Singapore, Singapore
  • [ 9 ] [Liang, Yuan]Singapore Management Univ, Sch Comp & Informat Syst, Singapore, Singapore
  • [ 10 ] [He, Shengfeng]Singapore Management Univ, Sch Comp & Informat Syst, Singapore, Singapore

Reprint 's Address:

Show more details

Related Keywords:

Related Article:

Source :

CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023)

ISSN: 1550-5499

Year: 2023

Page: 8964-8974

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 0

Online/Total:147/10033014
Address:FZU Library(No.2 Xuyuan Road, Fuzhou, Fujian, PRC Post Code:350116) Contact Us:0591-22865326
Copyright:FZU Library Technical Support:Beijing Aegean Software Co., Ltd. 闽ICP备05005463号-1