CRFormer: A cross-region transformer for shadow removal

Abstract

Image shadow removal is a fundamental task in computer vision, which aims to restore damaged signals caused by shadows, thereby improving image quality and scene understanding. Recently, transformers have demonstrated strong capabilities in various applications by capturing global pixel interactions, a capability highly desirable for shadow removal. However, applying transformers to promote shadow removal is non-trivial for the following two reasons: 1) The patchify operation is not suitable for shadow removal due to irregular shadow shapes; 2) Shadow removal only requires one-way interaction from the non-shadow region to the shadow region instead of the common two-way interactions among all pixels in the image. In this paper, we propose a novel Cross-Region transFormer (CRFormer) for shadow removal which differs from existing transformers by only considering the pixel interactions from the non-shadow region to the shadow region without splitting images into patches. This is achieved by a carefully designed region-aware cross-attention mechanism that aggregates the recovered shadow region features, conditioned on the non-shadow region features. Extensive experiments on the ISTD, AISTD, SRD, and Video Shadow Removal datasets demonstrate the superiority of our method compared to other state-of-the-art methods.