Image Factorization for Inverse Rendering

Abstract

Inverse Rendering is a core Computer Vision problem as it involves complete decomposition of newlinean image into its constituting atomic components. These components can be stand-alone analyzed or newlinesuitably modified and recombined to solve the required image analysis task or achieve the required newlinegenerative content. Rather than aiming for full decomposition, many applications only require decomposition newlineinto only a few factors which themselves are simple combinations of the underlying atomic newlinecomponents. This makes image factorization a critical first step in several computer vision and image newlineprocessing applications. This factorization could either be optically motivated like reflectance-shading newlinedecomposition, white-balancing, illumination spectra-separation etc. or semantically motivated like newlinestyle-content disentanglement, foreground-background matting etc. newlineIn this thesis, we focus on the former and present several image factorization solutions with an newlineaim to use it for a downstream image based rendering application. Initially, we assume Lambertian newlinereflection only under the classical image formation model inspired from the Retinex theory. Our newlinefirst solution in this category requires multiple images of the scene as input, which we then relax for newlineour second solution which works on the single image input. Afterwards, we propose a novel image newlineformation model based on the specularity of the image content and provide two solutions using the newlinelow light enhancement problem as the vehicle for empirical validation. Towards the end, a novel prior newlineinduction technique is also presented based on learnable concepts and its utility is shown by improving newlineresults of pre-existing state-of-the-art image decomposition networks. We conclude with a summary, newlinelimitations, future research directions and possible additional applications. The thesis is organized newlineinto four units respectively discussing the problem definition and significance; Lambertian reflection newlinebased Intrinsic Image Decomposition problem, specularity respecting novel illumination f