{"type":"ScholarlyArticle","title":"Margin Learning Embedded Prediction for Video Anomaly Detection with A Few Anomalies","authors":[{"name":"Wen Liu","slug":"wen-liu","given":"Wen","family":"Liu","affiliation":"ShanghaiTech University"},{"name":"Weixin Luo","slug":"weixin-luo","given":"Weixin","family":"Luo","affiliation":"ShanghaiTech University"},{"name":"Zhengxin Li","slug":"zhengxin-li","given":"Zhengxin","family":"Li","affiliation":"ShanghaiTech University"},{"name":"Peilin Zhao","slug":"peilin-zhao","given":"Peilin","family":"Zhao","affiliation":"Tencent AI Lab"},{"name":"Shenghua Gao","slug":"shenghua-gao","given":"Shenghua","family":"Gao","affiliation":"ShanghaiTech University"}],"abstract":"Classical semi-supervised video anomaly detection assumes that only normal data are available in the training set because of the rare and unbounded nature of anomalies. It is obviously, however, these infrequently observed abnormal events can actually help with the detection of identical or similar abnormal events, a line of thinking that motivates us to study open-set supervised anomaly detection with only a few types of abnormal observed events and many normal events available. Under the assumption that normal events can be well predicted, we propose a Margin Learning Embedded Prediction (MLEP) framework. There are three features in MLEP- based open-set supervised video anomaly detection: i) we customize a video prediction framework that favors the prediction of normal events and distorts the prediction of abnormal events; ii) The margin learning framework learns a more compact normal data distribution and enlarges the margin between normal and abnormal events. Since abnormal events are unbounded, our framework consequently helps with the detection of abnormal events, even for anomalies that have never been previously observed. Therefore, our framework is suitable for the open-set supervised anomaly detection setting; iii) our framework can readily handle both frame-level and video-level anomaly annotations. Considering that video-level anomaly detection is more easily annotated in practice and that anomaly detection with a few anomalies is a more practical setting, our work thus pushes the application of anomaly detection towards real scenarios. Extensive experiments validate the effectiveness of our framework for anomaly detection.","doi":"10.24963/ijcai.2019/419","url":"https://ijcai.org/proceedings/2019/419","pdf":"https://ijcai.org/proceedings/2019/0419.pdf","markdown":"https://ijcai.org/proceedings/2019/419.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/419","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":419,"pages":{"first":3023,"last":3030},"keywords":["Machine Learning: Classification","Computer Vision: Video: Events, Activities and Surveillance"],"videos":[],"references":[{"position":1,"raw":"[Ahmed et al., 2016] M. Ahmed, A. N. Mahmood, and J. Hu. A survey of network anomaly detection techniques. Journal of Network and Computer Applications, 2016.","title":"A survey of network anomaly detection techniques","authors":"M Ahmed, A N Mahmood, J Hu","year":"2016","venue":"Journal of Network and Computer Applications","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chandola et al., 2009] V. Chandola, A. Banerjee, and V. Kumar. Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3):15, 2009.","title":"Anomaly detection: A survey","authors":"V Chandola, A Banerjee, V Kumar","year":"2009","venue":"ACM computing surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cheng et al., 2016] D. Cheng, Y. Gong, et al. Person re- identification by multi-channel parts-based cnn with im- proved triplet loss function. In CVPR, 2016.","title":"Person reidentification by multi-channel parts-based cnn with improved triplet loss function","authors":"D Cheng, Y Gong","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Donahue et al., 2015] J. Donahue, L. A. Hendricks, et al. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, 2015.","title":"Long-term recurrent convolutional networks for visual recognition and description","authors":"J Donahue, L A Hendricks","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong et al., 2017] Q. Dong, S. Gong, and X Zhu. Class rectification hard mining for imbalanced deep learning. ICCV, 2017.","title":"Class rectification hard mining for imbalanced deep learning","authors":"Q Dong, S Gong, Zhu","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hasan et al., 2016] M. Hasan, J. Choi, et al. Learning tem- poral regularity in video sequences. In CVPR, 2016.","title":"Learning temporal regularity in video sequences","authors":"M Hasan, J Choi","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2018] C. Huang, Y. Li, C. Loy, and X. Tang. Deep imbalanced learning for face recognition and at- tribute prediction. ECCV, 2018.","title":"Deep imbalanced learning for face recognition and attribute prediction","authors":"C Huang, Y Li, C Loy, X Tang","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ienco et al., 2017] D. Ienco, R. G Pensa, and R. Meo. A semisupervised approach to the detection and characteri- zation of outliers in categorical data. IEEE TNNLS, 2017.","title":"A semisupervised approach to the detection and characterization of outliers in categorical data","authors":"D Ienco, R G Pensa, R Meo","year":"2017","venue":"IEEE TNNLS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ionescu et al., 2017] R. Ionescu, S. Smeureanu, B. Alexe, and M. Popescu. Unmasking the abnormal events in video. In ICCV, 2017.","title":"Unmasking the abnormal events in video","authors":"R Ionescu, S Smeureanu, B Alexe, M Popescu","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Isola et al., 2017] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with con- ditional adversarial networks. In CVPR, 2017.","title":"Image-to-image translation with conditional adversarial networks","authors":"Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A Efros","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2015] D. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"D Kingma, J Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krawczyk et al., 2014] B. Krawczyk, M. Woźniak, et al. Cost-sensitive decision tree ensembles for effective imbal- anced classification. Applied Soft Computing, 2014.","title":"Cost-sensitive decision tree ensembles for effective imbalanced classification","authors":"B Krawczyk, M Woźniak","year":"2014","venue":"Applied Soft Computing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li and Guo, 2007] Y. Li and L. Guo. An active learning based tcm-knn algorithm for supervised network intrusion detection. Computers & security, 2007.","title":"An active learning based tcm-knn algorithm for supervised network intrusion detection","authors":"Y Li, L Guo","year":"2007","venue":"Computers & security","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2011] S. Li, Z. Wang, G. Zhou, and S. Lee. Semi- supervised learning for imbalanced sentiment classifica- tion. In IJCAI, 2011.","title":"Semisupervised learning for imbalanced sentiment classification","authors":"S Li, Z Wang, G Zhou, S Lee","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2018] T. Lin, P. Goyal, et al. Focal loss for dense object detection. TPAMI, 2018.","title":"Focal loss for dense object detection","authors":"T Lin, P Goyal","year":"2018","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2018] W. Liu, W. Luo, D. Lian, and S. Gao. Fu- ture frame prediction for anomaly detection-a new base- line. In CVPR, 2018.","title":"Future frame prediction for anomaly detection-a new baseline","authors":"W Liu, W Luo, D Lian, S Gao","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[López et al., 2012] V. López, A. Fernández, Jose G Moreno-Torres, and F. Herrera. Analysis of preprocess- ing vs. cost-sensitive learning for imbalanced classifica- tion. open problems on intrinsic data characteristics. Ex- pert Systems with Applications, 39(7):6585-6608, 2012.","title":"Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. open problems on intrinsic data characteristics","authors":"V López, A Fernández, Jose G Moreno-Torres, F Herrera","year":"2012","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lu et al., 2013] C. Lu, J. Shi, and J. Jia. Abnormal event detection at 150 fps in matlab. In ICCV, 2013.","title":"Abnormal event detection at 150 fps in matlab","authors":"C Lu, J Shi, J Jia","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Luo et al., 2017a] W. Luo, W. Liu, and S. Gao. Remember- ing history with convolutional lstm for anomaly detection. In ICME, 2017.","title":"Remembering history with convolutional lstm for anomaly detection","authors":"Luo","year":"2017","venue":"ICME","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Luo et al., 2017b] W. Luo, W. Liu, and S. Gao. A revisit of sparse coding based anomaly detection in stacked rnn framework. In ICCV, 2017.","title":"A revisit of sparse coding based anomaly detection in stacked rnn framework","authors":"Luo","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mathieu et al., 2016] M. Mathieu, C. Couprie, and Y. Le- Cun. Deep multi-scale video prediction beyond mean square error. ICLR, 2016.","title":"Deep multi-scale video prediction beyond mean square error","authors":"M Mathieu, C Couprie, Y Le-Cun","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ren et al., 2018] M. Ren, W. Zeng, et al. Learning to reweight examples for robust deep learning. ICML, 2018.","title":"Learning to reweight examples for robust deep learning","authors":"M Ren, W Zeng","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer- Assisted Intervention, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sabokrou1 et al., 2018] M. Sabokrou1, M. Khalooei2, M. Fathy1, and E. Adeli. Adversarially learned one-class classifier for novelty detection. In CVPR, 2018.","title":"Adversarially learned one-class classifier for novelty detection","authors":"M Sabokrou1, M Khalooei2, M Fathy1, E Adeli","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Schroff et al., 2015] F. Schroff, D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recog- nition and clustering. In CVPR, 2015.","title":"Facenet: A unified embedding for face recognition and clustering","authors":"F Schroff, D Kalenichenko, J Philbin","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Shi et al., 2015] X. Shi, Z. Chen, et al. Convolutional lstm network: A machine learning approach for precipitation nowcasting. In NIPS, 2015.","title":"Convolutional lstm network: A machine learning approach for precipitation nowcasting","authors":"Shi","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Simonyan and Zisserman, 2014] K. Simonyan and A. Zis- serman. Two-stream convolutional networks for action recognition in videos. In NIPS, 2014.","title":"Two-stream convolutional networks for action recognition in videos","authors":"K Simonyan, A Zisserman","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sultani et al., 2018] W. Sultani, C. Chen, and M. Shah. Real-world anomaly detection in surveillance videos. In CVPR, 2018.","title":"Real-world anomaly detection in surveillance videos","authors":"W Sultani, C Chen, M Shah","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Tang et al., 2009] Y. Tang, Y. Zhang, N. V Chawla, and S. Krasser. Svms modeling for highly imbalanced clas- sification. IEEE Transactions on Systems, Man, and Cy- bernetics, Part B (Cybernetics), 39(1):281-288, 2009.","title":"Svms modeling for highly imbalanced classification","authors":"Y Tang, Y Zhang, N V Chawla, S Krasser","year":"2009","venue":"IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Tian et al., 2011] J. Tian, H. Gu, and W. Liu. Imbal- anced classification using support vector machine ensem- ble. Neural Computing and Applications, 2011.","title":"Imbalanced classification using support vector machine ensemble","authors":"J Tian, H Gu, W Liu","year":"2011","venue":"Neural Computing and Applications","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Villegas et al., 2017] R. Villegas, J. Yang, S. Hong, X. Lin, and H. Lee. Decomposing motion and content for natural video sequence prediction. ICLR, 2017.","title":"Decomposing motion and content for natural video sequence prediction","authors":"R Villegas, J Yang, S Hong, X Lin, H Lee","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Weinberger and Saul, 2009] K. Q Weinberger and L. K Saul. Distance metric learning for large margin nearest neighbor classification. JMLR, 2009.","title":"Distance metric learning for large margin nearest neighbor classification","authors":"K Weinberger, L K Saul","year":"2009","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yan et al., 2015] Y. Yan, M. Chen, et al. Deep learning for imbalanced multimedia data classification. In Multimedia (ISM). IEEE, 2015.","title":"Deep learning for imbalanced multimedia data classification","authors":"Y Yan, M Chen","year":"2015","venue":"Multimedia (ISM)","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhu et al., 2017] J. Zhu, T. Park, P. Isola, and A. A Efros. Unpaired image-to-image translation using cycle- consistent adversarial networks. In ICCV, 2017.","title":"Unpaired image-to-image translation using cycleconsistent adversarial networks","authors":"J Zhu, T Park, P Isola, A A Efros","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}