VideoCLIP is a contrastive approach to pre-train a unified model for zeroshot video and text understanding, without using any labels on downstream tasks. VideoCLIP trains a transformer for video and text by contrasting temporally overlapping positive video-text pairs with hard negatives from nearest neighbor retrieval. VideoCLIP provides state-of-the-art performance on a diverse series of downstream tasks, including sequence-level text-video retrieval, VideoQA, token-level action localization, and action segmentation, surpassing prior work, and in some cases even outperforming supervised approaches.
In this video, I will talk about the following: What can VideoCLIP do? What is the VideoCLIP architecture? How it is pretrained? How are positive and negative pairs obtained for pretraining VideoCLIP? How does VideoCLIP perform?
For more details, please look at arxiv.org/pdf/... and github.com/fac...
Xu, Hu, Gargi Ghosh, Po-Yao Huang, Dmytro Okhonko, Armen Aghajanyan, Florian Metze, Luke Zettlemoyer, and Christoph Feichtenhofer. "VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding." In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6787-6800. 2021.
Негізгі бет VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding
No video
Пікірлер