{"type":"ScholarlyArticle","title":"SEVEN: Deep Semi-supervised Verification Networks","authors":[{"name":"Vahid Noroozi","slug":"vahid-noroozi","given":"Vahid","family":"Noroozi","affiliation":"Department of Electrical Engineering and Computer Science , Northwestern University , Evanston , IL , USA"},{"name":"Lei Zheng","slug":"lei-zheng","given":"Lei","family":"Zheng","affiliation":"Department of Electrical Engineering and Computer Science , Northwestern University , Evanston , IL , USA"},{"name":"Sara Bahaadini","slug":"sara-bahaadini","given":"Sara","family":"Bahaadini","affiliation":"† Department of Computer Science , University of Illinois at Chicago , IL , USA; Computer Science and Engineering Department , Lehigh University , Bethlehem , PA , USA"},{"name":"Sihong Xie","slug":"sihong-xie","given":"Sihong","family":"Xie","affiliation":"Department of Electrical Engineering and Computer Science , Northwestern University , Evanston , IL , USA"},{"name":"Philip S. Yu","slug":"philip-s-yu","given":"Philip S.","family":"Yu","affiliation":"Department of Electrical Engineering and Computer Science , Northwestern University , Evanston , IL , USA"}],"abstract":"Verification determines whether two samples belong to the same class or not, and has important applications such as face and fingerprint verification, where thousands or millions of categories are present but each category has scarce labeled examples, presenting two major challenges for existing deep learning models. We propose a deep semi-supervised model named SEmi-supervised VErification Network (SEVEN) to address these challenges. The model consists of two complementary components. The generative component addresses the lack of supervision within each category by learning general salient structures from a large amount of data across categories.\r\nThe discriminative component exploits the learned general features to mitigate the lack of supervision within categories, and also directs the generative component to find more informative structures of the whole data manifold. The two components are tied together in SEVEN to allow an end-to-end training of the two components. Extensive experiments on four verification tasks demonstrate that SEVEN significantly outperforms other state-of-the-art deep semi-supervised techniques when labeled data are in short supply. Furthermore, SEVEN is competitive with fully supervised baselines trained with a larger amount of labeled data. It indicates the importance of the generative component in SEVEN.","doi":"10.24963/ijcai.2017/358","url":"https://ijcai.org/proceedings/2017/358","pdf":"https://ijcai.org/proceedings/2017/0358.pdf","markdown":"https://ijcai.org/proceedings/2017/358.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/358","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":358,"pages":{"first":2571,"last":2577},"keywords":["Machine Learning: Neural Networks","Machine Learning: Semi-Supervised Learning","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Bahaadini et al., 2017] Sara Bahaadini, Neda Rohani, Scott Coughlin, Michael Zevin, Vicky Kalogera, and Aggelos K Kat- saggelos. Deep multi-view models for glitch classification. Inter- national Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2017.","title":"Deep multi-view models for glitch classification","authors":"Sara Bahaadini, Neda Rohani, Scott Coughlin, Michael Zevin, Vicky Kalogera, Aggelos K Katsaggelos","year":"2017","venue":"International Conference on Acoustics, Speech, and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bell and Bala, 2015] Sean Bell and Kavita Bala. Learning visual similarity for product design with convolutional neural networks. ACM Transactions on Graphics (TOG), 34(4):98, 2015.","title":"Learning visual similarity for product design with convolutional neural networks","authors":"Bala ; Bell, Sean Bell, Kavita Bala","year":"2015","venue":"ACM Transactions on Graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bromley et al., 1993] Jane Bromley, James W Bentz, Léon Bottou, Isabelle Guyon, Yann LeCun, Cliff Moore, Eduard Säckinger, and Roopak Shah. Signature verification using a siamese time delay neural network. International Journal of Pattern Recogni- tion and Artificial Intelligence, 7(04):669-688, 1993.","title":"Signature verification using a siamese time delay neural network","authors":"Jane Bromley, James W Bentz, Léon Bottou, Isabelle Guyon, Yann Lecun, Cliff Moore, Eduard Säckinger, Roopak Shah","year":"1993","venue":"International Journal of Pattern Recognition and Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chopra et al., 2005] Sumit Chopra, Raia Hadsell, and Yann Le- Cun. Learning a similarity metric discriminatively, with applica- tion to face verification. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'05), volume 1, pages 539-546. IEEE, 2005.","title":"Learning a similarity metric discriminatively, with application to face verification","authors":"Sumit Chopra, Raia Hadsell, Yann Le-Cun","year":"2005","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR'05)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dauphin et al., 2015] Yann Dauphin, Harm de Vries, and Yoshua Bengio. Equilibrated adaptive learning rates for non-convex op- timization. In Advances in Neural Information Processing Sys- tems, pages 1504-1512, 2015.","title":"Equilibrated adaptive learning rates for non-convex optimization","authors":"Yann Dauphin, Harm De Vries, Yoshua Bengio","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dumoulin and Visin, 2016] Vincent Dumoulin and Francesco Visin. A guide to convolution arithmetic for deep learning. arXiv preprint arXiv:1603.07285, 2016.","title":"A guide to convolution arithmetic for deep learning","authors":"Vincent Dumoulin, Francesco Visin","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Galbally et al., 2015] Javier Galbally, Moises Diaz-Cabrera, Miguel A Ferrer, Marta Gomez-Barrero, Aythami Morales, and Julian Fierrez. On-line signature recognition through the combination of real dynamic data and synthetically generated static data. Pattern Recognition, 48(9):2921-2934, 2015.","title":"On-line signature recognition through the combination of real dynamic data and synthetically generated static data","authors":"Javier Galbally, Moises Diaz-Cabrera, Miguel A Ferrer, Marta Gomez-Barrero, Aythami Morales, Julian Fierrez","year":"2015","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hoffer and Ailon, 2016] Elad Hoffer and Nir Ailon. Semi- supervised deep learning by metric embedding. arXiv preprint arXiv:1611.01449, 2016.","title":"Semisupervised deep learning by metric embedding","authors":"Elad Hoffer, Nir Ailon","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hu et al., 2014] Junlin Hu, Jiwen Lu, and Yap-Peng Tan. Discrim- inative deep metric learning for face verification in the wild. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1875-1882, 2014.","title":"Discriminative deep metric learning for face verification in the wild","authors":"Junlin Hu, Jiwen Lu, Yap-Peng Tan","year":"2014","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2012] Gary Huang, Marwan Mattar, Honglak Lee, and Erik G Learned-Miller. Learning to align from scratch. In Advances in Neural Information Processing Systems, pages 764- 772, 2012.","title":"Learning to align from scratch","authors":"Gary Huang, Marwan Mattar, Honglak Lee, Erik G Learned- Miller","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hull, 1994] Jonathan J. Hull. A database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 16(5):550-554, 1994.","title":"A database for handwritten text recognition research","authors":"Jonathan J Hull","year":"1994","venue":"IEEE Transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by re- ducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.","title":"Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jain et al., 2016] Himanshu Jain, Yashoteja Prabhu, and Manik Varma. Extreme Multi-label Loss Functions for Recommenda- tion, Tagging, Ranking & Other Missing Label Applications. In SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2016.","title":"Extreme Multi-label Loss Functions for Recommendation, Tagging, Ranking & Other Missing Label Applications","authors":"Himanshu Jain, Yashoteja Prabhu, Manik Varma","year":"2016","venue":"SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma et al., 2014] Diederik P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems, pages 3581-3589, 2014.","title":"Semi-supervised learning with deep generative models","authors":"Shakir Diederik P Kingma, Danilo Mohamed, Max Jimenez Rezende, Welling","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Koch, 2015] Gregory Koch. Siamese neural networks for one-shot image recognition. PhD thesis, University of Toronto, 2015.","title":"Siamese neural networks for one-shot image recognition","authors":"Gregory Koch, Koch","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436-444, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey Hinton","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lee, 2013] Dong-Hyun Lee. Pseudo-label: The simple and effi- cient semi-supervised learning method for deep neural networks. In ICML Workshop on Challenges in Representation Learning, volume 3, page 2, 2013.","title":"Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks","authors":"Dong-Hyun Lee","year":"2013","venue":"ICML Workshop on Challenges in Representation Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2016] Yawei Li, Lizuo Jin, AK Qin, Changyin Sun, Yew Soon Ong, and Tong Cui. Semi-supervised auto-encoder based on manifold learning. In International Joint Conference on Neural Networks (IJCNN), pages 4032-4039, 2016.","title":"Semi-supervised auto-encoder based on manifold learning","authors":"Yawei Li, Lizuo Jin, Changyin Ak Qin, Yew Sun, Soon Ong, Tong Cui","year":"2016","venue":"International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Maaløe et al., 2016] Lars Maaløe, Casper Kaae Sønderby, Søren Kaae Sønderby, and Ole Winther. Auxiliary deep genera- tive models. In International Conference on Machine Learning (ICML), pages 1445-1453, 2016.","title":"Auxiliary deep generative models","authors":"Maaløe","year":"2016","venue":"International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Masci et al., 2011] Jonathan Masci, Ueli Meier, Dan Cires ¸an, and Jürgen Schmidhuber. Stacked convolutional auto-encoders for hierarchical feature extraction. Artificial Neural Networks and Machine Learning (ICANN), pages 52-59, 2011.","title":"Stacked convolutional auto-encoders for hierarchical feature extraction","authors":"Jonathan Masci, Ueli Meier, Dan Cires ¸an, Jürgen Schmidhuber","year":"2011","venue":"Artificial Neural Networks and Machine Learning (ICANN)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nair and Hinton, 2010] Vinod Nair and Geoffrey E Hinton. Recti- fied linear units improve restricted boltzmann machines. In Inter- national Conference on Machine Learning (ICML), pages 807- 814, 2010.","title":"Rectified linear units improve restricted boltzmann machines","authors":"Vinod Nair, Geoffrey E Hinton","year":"2010","venue":"International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Oh Song et al., 2016] Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese. Deep metric learning via lifted structured feature embedding. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4004-4012, 2016.","title":"Deep metric learning via lifted structured feature embedding","authors":"Hyun Oh Song, Yu Xiang, Stefanie Jegelka, Silvio Savarese","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rasmus et al., 2015] Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri Valpola, and Tapani Raiko. Semi-supervised learning with ladder networks. In Advances in Neural Informa- tion Processing Systems, pages 3546-3554, 2015.","title":"Semi-supervised learning with ladder networks","authors":"Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri Valpola, Tapani Raiko","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sun et al., 2014] Yi Sun, Yuheng Chen, Xiaogang Wang, and Xiaoou Tang. Deep learning face representation by joint identification-verification. In Advances in Neural Information Processing Systems, pages 1988-1996, 2014.","title":"Deep learning face representation by joint identification-verification","authors":"Yi Sun, Yuheng Chen, Xiaogang Wang, Xiaoou Tang","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sun et al., 2017] Yao Sun, Lejian Ren, Zhen Wei, Bin Liu, Yan- long Zhai, and Si Liu. A weakly-supervised method for makeup- invariant face verification. Pattern Recognition, 2017.","title":"A weakly-supervised method for makeupinvariant face verification","authors":"Yao Sun, Lejian Ren, Zhen Wei, Bin Liu, Yanlong Zhai, Si Liu","year":"2017","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wolf, 2014] Lior Wolf. Deepface: Closing the gap to human-level performance in face verification. In IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2014.","title":"Deepface: Closing the gap to human-level performance in face verification","authors":"Lior Wolf","year":"2014","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xie and Philip, 2017] Sihong Xie and S Yu Philip. Active zero- shot learning: a novel approach to extreme multi-labeled classi- fication. International Journal of Data Science and Analytics, pages 1-10, 2017.","title":"Active zeroshot learning: a novel approach to extreme multi-labeled classification","authors":"Philip ; Xie, Sihong Xie, S Yu, Philip","year":"2017","venue":"International Journal of Data Science and Analytics","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zagoruyko and Komodakis, 2015] Sergey Zagoruyko and Nikos Komodakis. Learning to compare image patches via convolu- tional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4353-4361, 2015.","title":"Learning to compare image patches via convolutional neural networks","authors":"Komodakis Zagoruyko, Sergey Zagoruyko, Nikos Komodakis","year":"2015","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zheng et al., 2017] Lei Zheng, Vahid Noroozi, and Philip S Yu. Joint deep modeling of users and items using reviews for rec- ommendation. In 10th ACM International Conference on Web Search and Data Mining (WSDM), pages 425-434. ACM, 2017.","title":"Joint deep modeling of users and items using reviews for recommendation","authors":"Lei Zheng, Vahid Noroozi, Philip S Yu","year":"2017","venue":"10th ACM International Conference on Web Search and Data Mining (WSDM)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}