Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
Ironieser authored Jun 1, 2023
1 parent 6969e91 commit 24a709d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Here is the official implementation for CVPR 2023 paper "Weakly Supervised Video
Sequential video understanding, as an emerging video understanding task, has driven lots of researchers’ attention because of its goal-oriented nature. This paper studies weakly supervised sequential video understanding where the accurate time-stamp level text-video alignment is not provided. We solve this task by borrowing ideas from CLIP. Specifically, we use a transformer to aggregate frame-level features for video representation and use a pre-trained text encoder to encode the texts corresponding to each action and the whole video, respectively. To model the correspondence between text and video, we propose a multiple granularity loss, where the video-paragraph contrastive loss enforces matching between the whole video and the complete script, and a fine-grained frame-sentence contrastive loss enforces the matching between each action and its description. As the frame-sentence correspondence is not available, we propose to use the fact that video actions happen sequentially in the temporal domain to generate pseudo frame-sentence correspondence and supervise the network training with the pseudo labels. Extensive experiments on video sequence verification and texttovideo matching show that our method outperforms baselines by a large margin, which validates the effectiveness of our proposed approach.
![](https://github.com/svip-lab/WeakSVR/blob/main/figs/sequence%20video.jpg)
## Video Presentation
<center><a href="https://www.bilibili.com/video/BV1B94y1S7oP?share_source=copy_web" target="_blank" style="color: #990000"> Bilibili </a></center> <br/>
<center><a href="https://www.youtube.com/watch?v=AqozSRYP7Pc" target="_blank" style="color: #990000"> Bilibili </a></center> <br/>
<center><a href="https://www.youtube.com/watch?v=AqozSRYP7Pc" target="_blank" style="color: #990000"> YouTube </a></center>

## Usage
Expand Down

0 comments on commit 24a709d

Please sign in to comment.