Guidefill: GPU accelerated, artist guided geometric inpainting for 3D conversion of film

DOI10.1137/16M1103737zbMATH Open1451.94009arXiv1611.05319MaRDI QIDQ4689656FDOQ4689656

Authors: L. Robert Hocking, Russell MacKenzie, Carola-Bibiane Schönlieb 

Publication date: 17 October 2018

Published in: SIAM Journal on Imaging Sciences (Search for Journal in Brave)

Abstract: The conversion of traditional film into stereo 3D has become an important problem in the past decade. One of the main bottlenecks is a disocclusion step, which in commercial 3D conversion is usually done by teams of artists armed with a toolbox of inpainting algorithms. A current difficulty in this is that most available algorithms are either too slow for interactive use, or provide no intuitive means for users to tweak the output. In this paper we present a new fast inpainting algorithm based on transporting along automatically detected splines, which the user may edit. Our algorithm is implemented on the GPU and fills the inpainting domain in successive shells that adapt their shape on the fly. In order to allocate GPU resources as efficiently as possible, we propose a parallel algorithm to track the inpainting interface as it evolves, ensuring that no resources are wasted on pixels that are not currently being worked on. Theoretical analysis of the time and processor complexiy of our algorithm without and with tracking (as well as numerous numerical experiments) demonstrate the merits of the latter. Our transport mechanism is similar to the one used in coherence transport, but improves upon it by corrected a "kinking" phenomena whereby extrapolated isophotes may bend at the boundary of the inpainting domain. Theoretical results explaining this phenomena and its resolution are presented. Although our method ignores texture, in many cases this is not a problem due to the thin inpainting domains in 3D conversion. Experimental results show that our method can achieve a visual quality that is competitive with the state-of-the-art while maintaining interactive speeds and providing the user with an intuitive interface to tweak the results.

Full work available at URL: https://arxiv.org/abs/1611.05319

Recommendations

• Fast image inpainting based on coherence transport
• Video inpainting of complex scenes
• Geodesic path based digital inpainting
• High efficiency depth image-based rendering with simplified inpainting-based hole filling
• A numerical algorithm for image sequence inpainting that preserves fine textures

zbMATH Keywords

image inpaintingimage processingparallel algorithmsGPUPDEs3D conversion

Mathematics Subject Classification ID

Numerical computation using splines (65D07) Image processing (compression, reconstruction, etc.) in information and communication theory (94A08) Computing methodologies for image processing (68U10) Parallel algorithms in computer science (68W10) Numerical algorithms for specific classes of architectures (65Y10)

Cites Work

• A well-posedness framework for inpainting based on coherence transport
• Partial differential equation methods for image inpainting
• GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems
• Title not available (Why is that?)
• Filling-in by joint interpolation of vector fields and gray levels
• Euler's Elastica and Curvature-Based Inpainting
• Cahn-Hilliard inpainting and a generalization for grayvalue images
• Variational image inpainting
• Fast image inpainting based on coherence transport
• A variational framework for exemplar-based image inpainting
• Image inpainting based on coherence transport with adapted distance functions
• A variational model for gradient-based video editing
• Geometrically guided exemplar-based inpainting
• Title not available (Why is that?)
• Depth-Guided Disocclusion Inpainting of Synthesized RGB-D Images
• Exemplar-based Inpainting: Technical Review and new Heuristics for better Geometric Reconstructions
• Video inpainting of complex scenes
• Space-Time Hole Filling With Random Walks in View Extrapolation for 3D Video

Cited In (1)

• Analysis of artifacts in shell-based image inpainting: why they occur and how to eliminate them

Uses Software

• CUDA
• inpaintBCT
• PatchMatch

This page was built for publication: Guidefill: GPU accelerated, artist guided geometric inpainting for 3D conversion of film

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4689656)