DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network (2207.10434v2)

Abstract: Shadow removal from a single image is generally still an open problem. Most existing learning-based methods use supervised learning and require a large number of paired images (shadow and corresponding non-shadow images) for training. A recent unsupervised method, Mask-ShadowGAN~\cite{Hu19}, addresses this limitation. However, it requires a binary mask to represent shadow regions, making it inapplicable to soft shadows. To address the problem, in this paper, we propose an unsupervised domain-classifier guided shadow removal network, DC-ShadowNet. Specifically, we propose to integrate a shadow/shadow-free domain classifier into a generator and its discriminator, enabling them to focus on shadow regions. To train our network, we introduce novel losses based on physics-based shadow-free chromaticity, shadow-robust perceptual features, and boundary smoothness. Moreover, we show that our unsupervised network can be used for test-time training that further improves the results. Our experiments show that all these novel components allow our method to handle soft shadows, and also to perform better on hard shadows both quantitatively and qualitatively than the existing state-of-the-art shadow removal methods. Our code is available at: \url{https://github.com/jinyeying/DC-ShadowNet-Hard-and-Soft-Shadow-Removal}.

• The paper introduces an unsupervised domain-classifier guided network that effectively removes both hard and soft shadows from single images.
• It leverages novel loss functions—physics-based chromaticity, perceptual feature, and boundary smoothness losses—to enhance shadow removal accuracy.
• Experimental results show significant improvements in RMSE and PSNR, outperforming state-of-the-art methods on challenging datasets.

Review of DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network

In the paper titled "DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network," the authors introduce an innovative method for shadow removal from single images that circumvents the limitations of existing techniques, particularly those reliant on supervised learning. The inadequacy of paired datasets for shadow and corresponding non-shadow images has been a significant barrier; this work tackles that via an unsupervised approach, employing a domain-classifier guided network architecture.

Methodology

The core contribution of this work is DC-ShadowNet, designed to proficiently address both hard and soft shadow removal. The network incorporates a domain classifier into both the generator and discriminator of a GAN, guiding focus on shadow versus shadow-free regions. This integration assists in effective shadow removal without excessive reliance on paired shadow/non-shadow datasets.

Novel losses introduced in this paper include:

• Physics-based shadow-free chromaticity loss: This leverages entropy minimization in the log-chromaticity space for generating a chromaticity map devoid of shadows, aligning the generator’s output with this map to refine shadow removal accuracy.
• Shadow-robust perceptual features loss: Derived from pre-trained VGG-16 features, this loss accounts for perceptual similarity between shadow and shadow-free regions, enhancing structural representation in shadow removal.
• Boundary smoothness loss: Ensures smooth transitions at shadow boundaries, thereby mitigating artifacts and harsh gradients post shadow removal.

Additionally, the unsupervised network incorporates test-time training, effectively allowing further performance enhancements when applied to testing datasets.

Experimental Results

The empirical results establish DC-ShadowNet's superior performance compared to state-of-the-art methods, both unsupervised like Mask-ShadowGAN and supervised methods. Notably, on the SRD and AISTD datasets, the proposed method shows substantially improved RMSE in shadow regions—33% and 18% better than Mask-ShadowGAN, respectively. For soft shadows, the method achieves significant improvement in RMSE and PSNR on the LRSS dataset, indicating robustness across diverse shadow conditions.

Implications and Future Directions

The implications for applications involving image editing and processing are profound given the enhanced applicability to mixed, realistic shadow environments without the necessity of extensive paired datasets. The proposed mechanism, particularly the domain-classifier guidance, could be further explored across varying domains of image denoising and relighting tasks.

Notwithstanding these strengths, future work could explore alternative unsupervised learning paradigms that might reduce dependency on explicit shadow-domain descriptors or enhance interpretability. Developing more efficient network architectures that maintain or enhance accuracy could bolster rapid adoption in real-time image processing applications.

In conclusion, DC-ShadowNet presents itself as a robust, unsupervised solution to the persistent challenge of shadow removal from single images, setting a benchmark in the domain of image restoration and processing. The proposed framework, with its novel fusion of domain guidance and physics-based losses, promises a valuable contribution to the field, paving the path for future explorations in domain-aligned unsupervised learning frameworks.