{"type":"ScholarlyArticle","title":"Batch Decorrelation for Active Metric Learning","authors":[{"name":"Priyadarshini Kumari","slug":"priyadarshini-kumari","given":"Priyadarshini","family":"Kumari","affiliation":"IIT Bombay"},{"name":"Ritesh Goru","slug":"ritesh-goru","given":"Ritesh","family":"Goru","affiliation":"IIT Bombay"},{"name":"Siddhartha Chaudhuri","slug":"siddhartha-chaudhuri","given":"Siddhartha","family":"Chaudhuri","affiliation":"IIT Bombay; Adobe Research"},{"name":"Subhasis Chaudhuri","slug":"subhasis-chaudhuri","given":"Subhasis","family":"Chaudhuri","affiliation":"IIT Bombay"}],"abstract":"We present an active learning strategy for training parametric models of distance metrics, given triplet-based similarity assessments: object $x_i$ is more similar to object $x_j$ than to $x_k$. In contrast to prior work on class-based learning, where the fundamental goal is classification and any implicit or explicit metric is binary, we focus on perceptual metrics that express the degree of (dis)similarity between objects. We find that standard active learning approaches degrade when annotations are requested for batches of triplets at a time: our studies suggest that correlation among triplets is responsible. In this work, we propose a novel method to decorrelate batches of triplets, that jointly balances informativeness and diversity while decoupling the choice of heuristic for each criterion. Experiments indicate our method is general, adaptable, and outperforms the state-of-the-art.","doi":"10.24963/ijcai.2020/312","url":"https://ijcai.org/proceedings/2020/312","pdf":"https://ijcai.org/proceedings/2020/0312.pdf","markdown":"https://ijcai.org/proceedings/2020/312.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/312","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":312,"pages":{"first":2255,"last":2261},"keywords":["Machine Learning: Active Learning","Machine Learning: Dimensionality Reduction and Manifold Learning","Machine Learning: Learning Preferences or Rankings"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25289/file_25289.m3u8","length_sec":298,"provider":"hls"}],"references":[{"position":1,"raw":"[Adomavicius and Tuzhilin, 2005] Gediminas Adomavicius and Alexander Tuzhilin. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. TKDE, 2005.","title":"Gediminas Adomavicius and Alexander Tuzhilin. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions","authors":"Tuzhilin Adomavicius","year":"2005","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arthur and Vassilvitskii, 2006] David Arthur and Sergei Vassilvitskii. k-means++: The advantages of careful seed- ing. Technical report, 2006.","title":"k-means++: The advantages of careful seeding","authors":"David Arthur, Sergei Vassilvitskii","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ash et al., 2020] Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In ICLR, 2020.","title":"Deep batch active learning by diverse, uncertain gradient lower bounds","authors":"Chicheng Jordan T Ash, Akshay Zhang, John Krishnamurthy, Alekh Langford, Agarwal","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bachman et al., 2017] Philip Bachman, Alessandro Sor- doni, and Adam Trischler. Learning algorithms for active learning. In ICML, 2017.","title":"Learning algorithms for active learning","authors":"Philip Bachman, Alessandro Sordoni, Adam Trischler","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Baldridge and Palmer, 2009] Jason Baldridge and Alexis Palmer. How well does active learning actually work?: Time-based evaluation of cost-reduction strategies for lan- guage documentation. In EMNLP, 2009.","title":"How well does active learning actually work?: Time-based evaluation of cost-reduction strategies for language documentation","authors":"Palmer ; Baldridge, Jason Baldridge, Alexis Palmer","year":"2009","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Calinon et al., 2007] Sylvain Calinon, Florent Guenter, and Aude Billard. On learning, representing, and generalizing a task in a humanoid robot. TSMCC, 2007.","title":"On learning, representing, and generalizing a task in a humanoid robot","authors":"Florent Sylvain Calinon, Aude Guenter, Billard","year":"2007","venue":"TSMCC","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Eldar et al., 1997] Yuval Eldar, Michael Lindenbaum, Moshe Porat, and Yehoshua Y Zeevi. The farthest point strategy for progressive image sampling. TIP, 1997.","title":"The farthest point strategy for progressive image sampling","authors":"Yuval Eldar, Michael Lindenbaum, Moshe Porat, Yehoshua Y Zeevi","year":"1997","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Freytag et al., 2014] Alexander Freytag, Erik Rodner, and Joachim Denzler. Selecting influential examples: Active learning with expected model output changes. In ECCV, 2014.","title":"Selecting influential examples: Active learning with expected model output changes","authors":"Alexander Freytag, Erik Rodner, Joachim Denzler","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gal et al., 2017] Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep Bayesian active learning with image data. In ICML, 2017.","title":"Deep Bayesian active learning with image data","authors":"Yarin Gal, Riashat Islam, Zoubin Ghahramani","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Golovin and Krause, 2011] Daniel Golovin and Andreas Krause. Adaptive submodularity: Theory and applications in active learning and stochastic optimization. JAIR, 2011.","title":"Adaptive submodularity: Theory and applications in active learning and stochastic optimization","authors":"Golovin, Krause, Daniel Golovin, Andreas Krause","year":"2011","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Heim et al., 2015] Eric Heim, Matthew Berger, Lee Sever- sky, and Milos Hauskrecht. Active perceptual similarity modeling with auxiliary information. In AAAI, 2015.","title":"Active perceptual similarity modeling with auxiliary information","authors":"Eric Heim, Matthew Berger, Lee Seversky, Milos Hauskrecht","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hoi et al., 2006] Steven CH Hoi, Rong Jin, Jianke Zhu, and Michael R Lyu. Batch mode active learning and its appli- cation to medical image classification. In ICML, 2006.","title":"Batch mode active learning and its application to medical image classification","authors":"C H Steven, Rong Hoi, Jianke Jin, Zhu, Michael R Lyu","year":"2006","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2016] Jiaji Huang, Rewon Child, Vinay Rao, Hairong Liu, Sanjeev Satheesh, and Adam Coates. Active learning for speech recognition: the power of gradients. In NeurIPSW, 2016.","title":"Active learning for speech recognition: the power of gradients","authors":"Huang","year":"2016","venue":"NeurIPSW","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kirsch et al., 2019] Andreas Kirsch, Joost van Amersfoort, and Yarin Gal. BatchBALD: Efficient and diverse batch acquisition for deep Bayesian active learning. In NeurIPS, 2019. [Konyushkova et al., 2017]","title":"BatchBALD: Efficient and diverse batch acquisition for deep Bayesian active learning","authors":"Andreas Kirsch, Joost Van Amersfoort, Yarin Gal","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"Ksenia Konyushkova, Raphael Sznitman, and Pascal Fua. Learning active learning from data. In NeurIPS, 2017.","title":"Learning active learning from data","authors":"Ksenia Konyushkova, Raphael Sznitman, Pascal Fua","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Krishnamurthy et al., 2017] Akshay Krishnamurthy, Alekh Agarwal, Tzu-Kuo Huang, Hal Daume III, and John Lang- ford. Active learning for cost-sensitive classification. In ICML, 2017.","title":"Active learning for cost-sensitive classification","authors":"Akshay Krishnamurthy, Alekh Agarwal, Tzu-Kuo Huang, Hal Daume, Iii, John Langford","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lohaus et al., 2019] Michael Lohaus, Philipp Hennig, and Ulrike von Luxburg. Uncertainty estimates for ordinal em- beddings. arXiv preprint arXiv:1906.11655, 2019.","title":"Uncertainty estimates for ordinal embeddings","authors":"Michael Lohaus, Philipp Hennig, Ulrike Von, Luxburg","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lu et al., 2017] Rui Lu, Kailun Wu, Zhiyao Duan, and Changshui Zhang. Deep ranking: Triplet MatchNet for music metric learning. In ICASSP, 2017.","title":"Deep ranking: Triplet MatchNet for music metric learning","authors":"Rui Lu, Kailun Wu, Zhiyao Duan, Changshui Zhang","year":"2017","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Priyadarshini K et al., 2019] Priyadarshini K, Siddhartha Chaudhuri, and Subhasis Chaudhuri. PerceptNet: Learn- ing perceptual similarity of haptic textures in presence of unorderable triplets. In WHC, 2019.","title":"PerceptNet: Learning perceptual similarity of haptic textures in presence of unorderable triplets","authors":"K Priyadarshini, Siddhartha Chaudhuri, Subhasis Chaudhuri","year":"2019","venue":"WHC","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sener and Savarese, 2018] Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core- set approach. In ICLR, 2018.","title":"Active learning for convolutional neural networks: A coreset approach","authors":"Ozan Sener, Silvio Savarese","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Settles, 2012] Burr Settles. Active learning. SLAIML, 2012.","title":"Active learning","authors":"Burr Settles","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sinha et al., 2019] Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. Variational adversarial active learning. In ICCV, 2019.","title":"Variational adversarial active learning","authors":"Samarth Sinha, Sayna Ebrahimi, Trevor Darrell","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Strese et al., 2017] M. Strese, Y. Boeck, and E. Steinbach. Content-based surface material retrieval. In WHC, 2017.","title":"Content-based surface material retrieval","authors":"M Strese, Y Boeck, E Steinbach","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tamuz et al., 2011] Omer Tamuz, Ce Liu, Serge Belongie, Ohad Shamir, and Adam Tauman Kalai. Adaptively learn- ing the crowd kernel. In ICML, 2011.","title":"Adaptively learning the crowd kernel","authors":"Omer Tamuz, Ce Liu, Serge Belongie, Ohad Shamir, Adam Tauman, Kalai","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wei et al., 2015] Kai Wei, Rishabh Iyer, and Jeff Bilmes. Submodularity in data subset selection and active learning. In ICML, 2015.","title":"Submodularity in data subset selection and active learning","authors":"Kai Wei, Rishabh Iyer, Jeff Bilmes","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wilber et al., 2014] Michael J Wilber, Iljung S Kwak, and Serge J Belongie. Cost-effective hits for relative similarity comparisons. In AAAI, 2014.","title":"Cost-effective hits for relative similarity comparisons","authors":"J Michael, Iljung S Wilber, Serge J Kwak, Belongie","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2017] Lin Yang, Yizhe Zhang, Jianxu Chen, Siyuan Zhang, and Danny Z Chen. Suggestive annotation: A deep active learning framework for biomedical image segmentation. In MICCAI, 2017.","title":"Suggestive annotation: A deep active learning framework for biomedical image segmentation","authors":"Lin Yang, Yizhe Zhang, Jianxu Chen, Siyuan Zhang, Danny Z Chen","year":"2017","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2017] Ye Zhang, Matthew Lease, and Byron C Wallace. Active discriminative text representation learn- ing. In AAAI, 2017.","title":"Active discriminative text representation learning","authors":"Zhang","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2018] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.","title":"The unreasonable effectiveness of deep features as a perceptual metric","authors":"Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, Oliver Wang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}