A simple framework to leverage state-of-the-art single-image super-resolution methods to restore light fields

Abstract

This paper describes a simple framework allowing us to leverage state-of-the-art single image super-resolution (SISR) techniques into light fields, while taking into account specific light field geometrical constraints. The idea is to first compute a representation compacting most of the light field energy into as few components as possible. This is achieved by aligning the light field using optical flow and then by decomposing the aligned light field using singular value decomposition (SVD). The principal basis captures the information that is coherent across all the views, while the other basis contain the high angular frequencies. Super-resolving this principal basis using an SISR method allows us to super-resolve all the information that is coherent across the entire light field. In this paper, to demonstrate the effectiveness of the approach, we have used the very deep super resolution (VDSR) method, which is one of the leading SISR algorithms, to restore the principal basis. The information restored in the principal basis is then propagated to restore all the other views using the computed optical flow. This framework allows the proposed light field super-resolution method to inherit the benefits of the SISR method used. Experimental results show that the proposed method is competitive, and most of the time superior, to recent light field super-resolution methods in terms of both PSNR and SSIM quality metrics, with a lower complexity. Moreover, the subjective results demonstrate that our method manages to restore sharper light fields which enables to generate refocused images of higher quality.