{"type":"ScholarlyArticle","title":"CTW: Confident Time-Warping for Time-Series Label-Noise Learning","authors":[{"name":"Peitian Ma","slug":"peitian-ma","given":"Peitian","family":"Ma","affiliation":"School of Computer Science and Engineering , South China University of Technology , Guangzhou , China"},{"name":"Zhen Liu","slug":"zhen-liu","given":"Zhen","family":"Liu","affiliation":"School of Computer Science and Engineering , South China University of Technology , Guangzhou , China"},{"name":"Junhao Zheng","slug":"junhao-zheng","given":"Junhao","family":"Zheng","affiliation":"School of Computer Science and Engineering , South China University of Technology , Guangzhou , China"},{"name":"Linghao Wang","slug":"linghao-wang","given":"Linghao","family":"Wang","affiliation":"School of Computer Science and Engineering , South China University of Technology , Guangzhou , China"},{"name":"Qianli Ma","slug":"qianli-ma","given":"Qianli","family":"Ma","affiliation":"School of Computer Science and Engineering , South China University of Technology , Guangzhou , China; Key Laboratory of Big Data and Intelligent Robot ( South China University of Technology) , Ministry of Education ma"}],"abstract":"Noisy labels seriously degrade the generalization ability of Deep Neural Networks (DNNs) in various classification tasks. Existing studies on label-noise learning mainly focus on computer vision, while time series also suffer from the same issue. Directly applying the methods from computer vision to time series may reduce the temporal dependency due to different data characteristics. How to make use of the properties of time series to enable DNNs to learn robust representations in the presence of noisy labels has not been fully explored. To this end, this paper proposes a method that expands the distribution of Confident instances by Time-Warping (CTW) to learn robust representations of time series. Specifically, since applying the augmentation method to all data may introduce extra mislabeled data, we select confident instances to implement Time-Warping. In addition, we normalize the distribution of the training loss of each class to eliminate the model's selection preference for instances of different classes, alleviating the class imbalance caused by sample selection. Extensive experimental results show that CTW achieves state-of-the-art performance on the UCR datasets when dealing with different types of noise. Besides, the t-SNE visualization of our method verifies that augmenting confident data improves the generalization ability. Our code is available at https://github.com/qianlima-lab/CTW.","doi":"10.24963/ijcai.2023/450","url":"https://ijcai.org/proceedings/2023/450","pdf":"https://ijcai.org/proceedings/2023/0450.pdf","markdown":"https://ijcai.org/proceedings/2023/450.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/450","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":450,"pages":{"first":4046,"last":4054},"keywords":["Machine Learning: ML: Representation learning","Machine Learning: ML: Time series and data streams","Machine Learning: ML: Classification"],"videos":[],"references":[{"position":1,"raw":"[Arazo et al., 2019] Eric Arazo, Diego Ortego, Paul Albert, Noel O'Connor, and Kevin McGuinness. Unsupervised label noise modeling and loss correction. In International conference on machine learning, pages 312-321. PMLR, 2019.","title":"Unsupervised label noise modeling and loss correction","authors":"Eric Arazo, Diego Ortego, Paul Albert, O' Noel, Kevin Connor, Mcguinness","year":"2019","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arpit et al., 2017] Devansh Arpit, Stanisław Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memo- rization in deep networks. In International conference on machine learning, pages 233-242. PMLR, 2017.","title":"A closer look at memorization in deep networks","authors":"Devansh Arpit, Stanisław Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, S Maxinder, Tegan Kanwal, Asja Maharaj, Aaron Fischer, Yoshua Courville, Bengio","year":"2017","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Atkinson and Metsis, 2020] Gentry Atkinson and Vangelis Metsis. Identifying label noise in time-series datasets. In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers, pages 238-243, 2020.","title":"Identifying label noise in time-series datasets","authors":"Gentry Atkinson, Vangelis Metsis","year":"2020","venue":"Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bagnall et al., 2018] Anthony Bagnall, Hoang Anh Dau, Ja- son Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. The uea multivari- ate time series classification archive, 2018. arXiv preprint arXiv:1811.00075, 2018.","title":"The uea multivariate time series classification archive","authors":"Anthony Bagnall, Anh Hoang, Jason Dau, Michael Lines, James Flynn, Aaron Large, Paul Bostrom, Eamonn Southam, Keogh","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Berthelot et al., 2019] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi- supervised learning. Advances in neural information pro- cessing systems, 32, 2019.","title":"Mixmatch: A holistic approach to semisupervised learning","authors":"David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin A Raffel","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Castellani et al., 2021] Andrea Castellani, Sebastian Schmitt, and Barbara Hammer. Estimating the electri- cal power output of industrial devices with end-to-end time-series classification in the presence of label noise. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 469-484.","title":"Estimating the electrical power output of industrial devices with end-to-end time-series classification in the presence of label noise","authors":"Andrea Castellani, Sebastian Schmitt, Barbara Hammer","year":null,"venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2019] Pengfei Chen, Ben Ben Liao, Guangyong Chen, and Shengyu Zhang. Understanding and utilizing deep neural networks trained with noisy labels. In Inter- national Conference on Machine Learning, pages 1062- 1070. PMLR, 2019.","title":"Understanding and utilizing deep neural networks trained with noisy labels","authors":"Pengfei Chen, Ben Ben Liao, Guangyong Chen, Shengyu Zhang","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cheng et al., 2020] Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. Learning with instance-dependent label noise: A sample sieve approach. arXiv preprint arXiv:2010.02347, 2020.","title":"Learning with instance-dependent label noise: A sample sieve approach","authors":"Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, Yang Liu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dau et al., 2019] Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn Keogh. The ucr time series archive. IEEE/CAA Journal of Automatica Sinica, 6(6):1293-1305, 2019.","title":"The ucr time series archive","authors":"Anh Hoang, Anthony Dau, Kaveh Bagnall, Chin-Chia Kamgar, Yan Michael Yeh, Shaghayegh Zhu, Chotirat Gharghabi, Ann Ratanamahatana, Eamonn Keogh","year":"2019","venue":"IEEE/CAA Journal of Automatica Sinica","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Demšar, 2006] Janez Demšar. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine learning research, 7:1-30, 2006.","title":"Janez Demšar. Statistical comparisons of classifiers over multiple data sets","authors":"Demšar","year":"2006","venue":"The Journal of Machine learning research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gui et al., 2021] Xian-Jin Gui, Wei Wang, and Zhang- Hao Tian. Towards understanding deep learning from noisy labels with small-loss criterion. arXiv preprint arXiv:2106.09291, 2021.","title":"Towards understanding deep learning from noisy labels with small-loss criterion","authors":"Xian-Jin Gui, Wei Wang, Zhang-Hao Tian","year":"2021","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2021/340"},{"position":13,"raw":"[Han et al., 2018] Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31, 2018.","title":"Co-teaching: Robust training of deep neural networks with extremely noisy labels","authors":"Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, Masashi Sugiyama","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Han et al., 2020] Bo Han, Gang Niu, Xingrui Yu, Quan- ming Yao, Miao Xu, Ivor Tsang, and Masashi Sugiyama. Sigua: Forgetting may make learning with noisy labels more robust. In International Conference on Machine Learning, pages 4006-4016. PMLR, 2020.","title":"Sigua: Forgetting may make learning with noisy labels more robust","authors":"Bo Han, Gang Niu, Xingrui Yu, Quanming Yao, Miao Xu, Ivor Tsang, Masashi Sugiyama","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hu et al., 2021] Zijian Hu, Zhengyu Yang, Xuefeng Hu, and Ram Nevatia. Simple: similar pseudo label exploita- tion for semi-supervised classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 15099-15108, 2021.","title":"Simple: similar pseudo label exploitation for semi-supervised classification","authors":"Zijian Hu, Zhengyu Yang, Xuefeng Hu, Ram Nevatia","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Huang et al., 2022] Yingsong Huang, Bing Bai, Shengwei Zhao, Kun Bai, and Fei Wang. Uncertainty-aware learn- ing against label noise on imbalanced datasets. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6960-6969, 2022.","title":"Uncertainty-aware learning against label noise on imbalanced datasets","authors":"Huang","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. In International conference on machine learning, pages 448-456. PMLR, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Jiang et al., 2018] Lu Jiang, Zhengyuan Zhou, Thomas Le- ung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data- driven curriculum for very deep neural networks on cor- rupted labels. In International conference on machine learning, pages 2304-2313. PMLR, 2018.","title":"Mentornet: Learning datadriven curriculum for very deep neural networks on corrupted labels","authors":"Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, Li Fei-Fei","year":"2018","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Karim et al., 2022] Nazmul Karim, Mamshad Nayeem Rizve, Nazanin Rahnavard, Ajmal Mian, and Mubarak Shah. Unicon: Combating label noise through uniform selection and contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9676-9686, 2022.","title":"Unicon: Combating label noise through uniform selection and contrastive learning","authors":"Nazmul Karim, Mamshad Nayeem Rizve, Nazanin Rahnavard, Ajmal Mian, Mubarak Shah","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Le Guennec et al., 2016] Arthur Le Guennec, Simon Mali- nowski, and Romain Tavenard. Data augmentation for time series classification using convolutional neural net- works. In ECML/PKDD workshop on advanced analytics and learning on temporal data, 2016.","title":"Arthur Le Guennec, Simon Malinowski, and Romain Tavenard. Data augmentation for time series classification using convolutional neural networks","authors":"Le Guennec","year":"2016","venue":"ECML/PKDD workshop on advanced analytics and learning on temporal data","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Li et al., 2020] Junnan Li, Richard Socher, and Steven CH Hoi. Dividemix: Learning with noisy labels as semi- supervised learning. arXiv preprint arXiv:2002.07394, 2020.","title":"Dividemix: Learning with noisy labels as semisupervised learning","authors":"Li","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Li et al., 2022] Shikun Li, Xiaobo Xia, Shiming Ge, and Tongliang Liu. Selective-supervised contrastive learn- ing with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 316-325, 2022.","title":"Selective-supervised contrastive learning with noisy labels","authors":"Shikun Li, Xiaobo Xia, Shiming Ge, Tongliang Liu","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Nishi et al., 2021] Kento Nishi, Yi Ding, Alex Rich, and To- bias Hollerer. Augmentation strategies for learning with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8022- 8031, 2021.","title":"Augmentation strategies for learning with noisy labels","authors":"Kento Nishi, Yi Ding, Alex Rich, Tobias Hollerer","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Reed et al., 2014] Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. Training deep neural networks on noisy la- bels with bootstrapping. arXiv preprint arXiv:1412.6596, 2014.","title":"Training deep neural networks on noisy labels with bootstrapping","authors":"Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, Andrew Rabinovich","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Song et al., 2019] Hwanjun Song, Minseok Kim, Dong- min Park, and Jae-Gil Lee. How does early stopping help generalization against label noise? arXiv preprint arXiv:1911.08059, 2019.","title":"How does early stopping help generalization against label noise","authors":"Song","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Song et al., 2022] Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: A survey. IEEE Trans- actions on Neural Networks and Learning Systems, 2022.","title":"Learning from noisy labels with deep neural networks: A survey","authors":"Song","year":"2022","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Um et al., 2017] Terry T Um, Franz MJ Pfister, Daniel Pich- ler, Satoshi Endo, Muriel Lang, Sandra Hirche, Urban Fi- etzek, and Dana Kulić. Data augmentation of wearable sensor data for parkinson's disease monitoring using con- volutional neural networks. In Proceedings of the 19th ACM international conference on multimodal interaction, pages 216-220, 2017.","title":"Data augmentation of wearable sensor data for parkinson's disease monitoring using convolutional neural networks","authors":"Terry T Um, M J Franz, Daniel Pfister, Satoshi Pichler, Muriel Endo, Sandra Lang, Urban Hirche, Dana Fietzek, Kulić","year":"2017","venue":"Proceedings of the 19th ACM international conference on multimodal interaction","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Verma et al., 2019] Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez- Paz, and Yoshua Bengio. Manifold mixup: Better rep- resentations by interpolating hidden states. In Interna- tional Conference on Machine Learning, pages 6438- 6447. PMLR, 2019.","title":"Manifold mixup: Better representations by interpolating hidden states","authors":"Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez-Paz, Yoshua Bengio","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wei et al., 2020] Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. Combating noisy labels by agreement: A joint training method with co-regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 13726-13735, 2020.","title":"Combating noisy labels by agreement: A joint training method with co-regularization","authors":"Hongxin Wei, Lei Feng, Xiangyu Chen, Bo An","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xia et al., 2019] Xiaobo Xia, Tongliang Liu, Nannan Wang, Bo Han, Chen Gong, Gang Niu, and Masashi Sugiyama. Are anchor points really indispensable in label-noise learn- ing? Advances in Neural Information Processing Systems, 32, 2019.","title":"Are anchor points really indispensable in label-noise learning","authors":"Xiaobo Xia, Tongliang Liu, Nannan Wang, Bo Han, Chen Gong, Gang Niu, Masashi Sugiyama","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xia et al., 2020] Xiaobo Xia, Tongliang Liu, Bo Han, Nan- nan Wang, Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. Part-dependent label noise: Towards instance-dependent label noise. Advances in Neural Information Processing Systems, 33:7597-7610, 2020.","title":"Part-dependent label noise: Towards instance-dependent label noise","authors":"Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang, Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, Masashi Sugiyama","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yao et al., 2020] Yu Yao, Tongliang Liu, Bo Han, Ming- ming Gong, Jiankang Deng, Gang Niu, and Masashi Sugiyama. Dual t: Reducing estimation error for transi- tion matrix in label-noise learning. Advances in neural information processing systems, 33:7260-7271, 2020.","title":"Dual t: Reducing estimation error for transition matrix in label-noise learning","authors":"Yu Yao, Tongliang Liu, Bo Han, Mingming Gong, Jiankang Deng, Gang Niu, Masashi Sugiyama","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yu et al., 2019] Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. How does dis- agreement help generalization against label corruption? In International Conference on Machine Learning, pages 7164-7173. PMLR, 2019.","title":"How does disagreement help generalization against label corruption?","authors":"Yu","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}