{"type":"ScholarlyArticle","title":"Self-Supervised Video Action Localization with Adversarial Temporal Transforms","authors":[{"name":"Guoqiang Gong","slug":"guoqiang-gong","given":"Guoqiang","family":"Gong","affiliation":""},{"name":"Liangfeng Zheng","slug":"liangfeng-zheng","given":"Liangfeng","family":"Zheng","affiliation":""},{"name":"Wenhao Jiang","slug":"wenhao-jiang","given":"Wenhao","family":"Jiang","affiliation":""},{"name":"Yadong Mu","slug":"yadong-mu","given":"Yadong","family":"Mu","affiliation":""}],"abstract":"Weakly-supervised temporal action localization aims to locate intervals of action instances with only video-level action labels for training. However, the localization results generated from video classification networks are often not accurate due to the lack of temporal boundary annotation of actions. Our motivating insight is that the temporal boundary of action should be stably predicted under various temporal transforms. This inspires a self-supervised equivariant transform consistency constraint. We design a set of temporal transform operations, including naive temporal down-sampling to learnable attention-piloted time warping. In our model, a localization network aims to perform well under all transforms, and another policy network is designed to choose a temporal transform at each iteration that adversarially brings localization result inconsistent with the localization network's. Additionally, we devise a self-refine module to enhance the completeness of action intervals harnessing temporal and semantic contexts. Experimental results on THUMOS14 and ActivityNet demonstrate that our model consistently outperforms the state-of-the-art weakly-supervised temporal action localization methods.","doi":"10.24963/ijcai.2021/96","url":"https://ijcai.org/proceedings/2021/96","pdf":"https://ijcai.org/proceedings/2021/0096.pdf","markdown":"https://ijcai.org/proceedings/2021/96.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/96","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Computer Vision I","track":"Main Track","number":96,"pages":{"first":693,"last":699},"keywords":["Computer Vision: Action Recognition","Computer Vision: Video: Events, Activities and Surveillance"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}