We present a new additive image factorization technique that treats images to be composed of multiple latent specular components that can be simply estimated recursively by modulating the sparsity during decomposition. Our model-driven RSFNet estimates these factors by unrolling the optimization into network layers requiring only a few scalars to be learned. The resultant factors are interpretable by design and can be fused for different image enhancement tasks via a network or combined directly by the user in a controllable fashion. Based on RSFNet, we detail a zero-reference Low Light Enhancement (LLE) application trained without paired or unpaired supervision. Our system improves the state-of-the-art performance on standard benchmarks and achieves better generalization on multiple other datasets. We also integrate our factors with other task specific fusion networks for applications like deraining, deblurring and dehazing with negligible overhead thereby highlighting the multi-domain and multi-task generalizability of our proposed RSFNet.
Our Recursive Specularity Factorization Network (RSFNet) decomposes the image into K factors using sequential Factorization Modules (FM), wherein each optimization step encoded as a differentiable network layer. Then we fuse, enhance, and denoise the factors using off-the-shelf task-dependent fusion network.
Our factors can be used as preprocessed structural priors for various image enhancement applications like deraining, dehazing and deblurring by simply substituting the input with the concatenated tensor and modifying the input channels appropriately.
@InProceedings{saini2024rsfnet,
author = {Saini, Saurabh and Narayanan, P. J.},
title = {Specularity Factorization for Low Light Enhancement},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2024},
}