A Unified framework based on Large-scale Momentum Contrastive learning for Text-Video Retrieval

Large-scale vision-language representation learning has shown dramatic advances in image-text retrieval studies. On top of that, it is clear that the CLIP (Contrastive Language-Image Pre-training) [1] has contributed to this field in that it succeeded pre-training on tremendous web multimedia data with leveraging contrastive loss, one of the most competitive loss for multimodal learning. CLIP4CLIP [2] utilized the pre-trained CLIP weight to appeal its performance of learning video representations with textual supervision with additional post-pretraining on Howto100M [3]. However, there is still room for elevation in CLIP4Clip from perspective of contrastive learning - large enough instances of negative samples help to learn better representations, as the batch size in each training batch getting larger, the more chance of the model to be allowed to contrast more semantic representations. In the same queue of this point of view, we propose CwMTVR combining Momentum Contrast (MoCo) [4], cross-modal momentum contrastive learning framework into CLIP4Clip. Experimental results prove that the CwMTVR model achieve competitive results in text-video retrieval on MSVD dataset.