{"type":"ScholarlyArticle","title":"Simultaneous Clustering and Ranking from Pairwise Comparisons","authors":[{"name":"Jiyi Li","slug":"jiyi-li","given":"Jiyi","family":"Li","affiliation":"University of Yamanashi , Japan; RIKEN Center for AIP , Japan"},{"name":"Yukino Baba","slug":"yukino-baba","given":"Yukino","family":"Baba","affiliation":"University of Tsukuba , Japan; RIKEN Center for AIP , Japan"},{"name":"Hisashi Kashima","slug":"hisashi-kashima","given":"Hisashi","family":"Kashima","affiliation":"Kyoto University , Japan; RIKEN Center for AIP , Japan"}],"abstract":"When people make decisions with a number of ideas, designs, or other kinds of objects, one attempt is probably to organize them into several groups of objects and to prioritize them according to some preference. The grouping task is referred to as clustering and the prioritizing task is called as ranking.\r\nThese tasks are often outsourced with the help of human judgments in the form of pairwise comparisons. Two objects are compared on whether they are similar in the clustering problem, while the object of higher priority is determined in the ranking problem. Our research question in this paper is whether the pairwise comparisons for clustering also help ranking (and vice versa). Instead of solving the two tasks separately, we propose a unified formulation to bridge the two types of pairwise comparisons. Our formulation simultaneously estimates the object embeddings and the preference criterion vector. The experiments using real datasets support our hypothesis; our approach can generate better neighbor and preference estimation results than the approaches that only focus on a single type of pairwise comparisons.","doi":"10.24963/ijcai.2018/215","url":"https://ijcai.org/proceedings/2018/215","pdf":"https://ijcai.org/proceedings/2018/0215.pdf","markdown":"https://ijcai.org/proceedings/2018/215.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/215","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Humans and AI","track":"Main track","number":215,"pages":{"first":1554,"last":1560},"keywords":["Humans and AI: Human Computation and Crowdsourcing","Machine Learning: Learning Preferences or Rankings","Machine Learning: Clustering"],"videos":[],"references":[{"position":1,"raw":"[Agarwal et al., 2007] Sameer Agarwal, Josh Wills, Lawrence Cayton, Gert Lanckriet, David Kriegman, and Serge Belongie. Generalized non-metric multidimensional scaling. In Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics, pages 11-18, 2007.","title":"Generalized non-metric multidimensional scaling","authors":"Sameer Agarwal, Josh Wills, Lawrence Cayton, Gert Lanckriet, David Kriegman, Serge Belongie","year":"2007","venue":"Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bradley and Terry, 1952] Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324- 345, 1952.","title":"Rank analysis of incomplete block designs: I. the method of paired comparisons","authors":"Terry ; Bradley, Allan Ralph, Milton E Bradley, Terry","year":"1952","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cattelan, 2012] Manuela Cattelan. Models for paired com- parison data: A review with emphasis on dependent data. Statistical Science, pages 412-433, 2012.","title":"Models for paired comparison data: A review with emphasis on dependent data","authors":"Manuela Cattelan","year":"2012","venue":"Statistical Science","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Causeur and Husson, 2005] David Causeur and Franc ¸ois Husson. A 2-dimensional extension of the bradley-terry model for paired comparisons. Journal of Statistical Plan- ning and Inference, 135(2):245-259, 2005.","title":"David Causeur and Franc ¸ois Husson. A 2-dimensional extension of the bradley-terry model for paired comparisons","authors":"Husson Causeur","year":"2005","venue":"Journal of Statistical Planning and Inference","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen and Joachims, 2016] Shuo Chen and Thorsten Joachims. Modeling intransitivity in matchup and com- parison data. In Proceedings of the 9th ACM International Conference on Web Search and Data Mining, pages 227-236, 2016.","title":"Modeling intransitivity in matchup and comparison data","authors":"Shuo Chen, Thorsten Joachims","year":"2016","venue":"Proceedings of the 9th ACM International Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2013] Xi Chen, Paul N. Bennett, Kevyn Collins-Thompson, and Eric Horvitz. Pairwise ranking aggregation in a crowdsourced setting. In Proceedings of the 6th ACM International Conference on Web Search and Data Mining, pages 193-202, 2013.","title":"Pairwise ranking aggregation in a crowdsourced setting","authors":"Xi Chen, Paul N Bennett, Kevyn Collins-Thompson, Eric Horvitz","year":"2013","venue":"Proceedings of the 6th ACM International Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gomes et al., 2011] Ryan G. Gomes, Peter Welinder, An- dreas Krause, and Pietro Perona. Crowdclustering. In Advances in Neural Information Processing Systems 24, pages 558-566. 2011.","title":"Crowdclustering","authors":"Ryan G Gomes, Peter Welinder, Andreas Krause, Pietro Perona","year":"2011","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hinton and Roweis, 2003] Geoffrey E Hinton and Sam T. Roweis. Stochastic neighbor embedding. In Advances in Neural Information Processing Systems 15, pages 857- 864. 2003.","title":"Stochastic neighbor embedding","authors":"Geoffrey E Hinton, Sam T Roweis","year":"2003","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hope et al., 2017] Tom Hope, Joel Chan, Aniket Kittur, and Dafna Shahaf. Accelerating innovation through analogy mining. In Proceedings of the 23rd ACM SIGKDD Inter- national Conference on Knowledge Discovery and Data Mining, pages 235-243, 2017.","title":"Accelerating innovation through analogy mining","authors":"Tom Hope, Joel Chan, Aniket Kittur, Dafna Shahaf","year":"2017","venue":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":"/proceedings/2018/736"},{"position":10,"raw":"[Khosla et al., 2011] Aditya Khosla, Nityananda Jayadevap- rakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine-grained image categorization. In Proceedings of the 1st Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recog- nition, 2011.","title":"Novel dataset for fine-grained image categorization","authors":"Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, Li Fei-Fei","year":"2011","venue":"Proceedings of the 1st Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Korlakai Vinayak and Hassibi, 2016] Ramya Kor- lakai Vinayak and Babak Hassibi. Crowdsourced clustering: Querying edges vs triangles. In Advances in Neural Information Processing Systems 29, pages 1316-1324. 2016.","title":"Crowdsourced clustering: Querying edges vs triangles","authors":"Korlakai Vinayak, ; Hassibi, Korlakai Ramya, Babak Vinayak, Hassibi","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Raman and Joachims, 2014] Karthik Raman and Thorsten Joachims. Methods for ordinal peer grading. In Proceed- ings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1037- 1046, 2014.","title":"Methods for ordinal peer grading","authors":"Karthik Raman, Thorsten Joachims","year":"2014","venue":"Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Siangliulue et al., 2015] Pao Siangliulue, Kenneth C. Arnold, Krzysztof Z. Gajos, and Steven P. Dow. Toward collaborative ideation at scale: Leveraging ideas from others to generate more creative and diverse ideas. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work, pages 937-945, 2015.","title":"Toward collaborative ideation at scale: Leveraging ideas from others to generate more creative and diverse ideas","authors":"Pao Siangliulue, Kenneth C Arnold, Krzysztof Z Gajos, Steven P Dow","year":"2015","venue":"Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Tamuz et al., 2011] Omer Tamuz, Ce Liu, Serge Belongie, Ohad Shamir, and Adam Tauman Kalai. Adaptively learn- ing the crowd kernel. In Proceedings of the 28th Inter- national Conference on International Conference on Ma- chine Learning, pages 673-680, 2011.","title":"Adaptively learning the crowd kernel","authors":"Omer Tamuz, Ce Liu, Serge Belongie, Ohad Shamir, Adam Tauman, Kalai","year":"2011","venue":"Proceedings of the 28th International Conference on International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ukkonen et al., 2015] Antti Ukkonen, Behrouz Der- akhshan, and Hannes Heikinheimo. Crowdsourced nonparametric density estimation using relative distances. In Proceedings of the 3rd AAAI Conference on Human Computation and Crowdsourcing, 2015.","title":"Crowdsourced nonparametric density estimation using relative distances","authors":"Ukkonen","year":"2015","venue":"Proceedings of the 3rd AAAI Conference on Human Computation and Crowdsourcing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[van der Maaten and Hinton, 2008] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Jour- nal of Machine Learning Research, 9(Nov):2579-2605, 2008.","title":"Visualizing data using t-SNE","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[van Der Maaten and Weinberger, 2012] Laurens van Der Maaten and Kilian Weinberger. Stochastic triplet embedding. In Proceedings of 2012 IEEE International Workshop on Machine Learning for Signal Processing, pages 1-6, 2012.","title":"Stochastic triplet embedding","authors":"Weinberger ; Laurens Van Der Maaten, Kilian Van Der Maaten, Weinberger","year":"2012","venue":"Proceedings of 2012 IEEE International Workshop on Machine Learning for Signal Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wah et al., 2014] Catherine Wah, Grant Van Horn, Steve Branson, Subhransu Maji, Pietro Perona, and Serge Be- longie. Similarity comparisons for interactive fine-grained categorization. In Proceedings of the 2014 IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 859-866, 2014.","title":"Similarity comparisons for interactive fine-grained categorization","authors":"Catherine Wah, Grant Van Horn, Steve Branson, Subhransu Maji, Pietro Perona, Serge Belongie","year":"2014","venue":"Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xie et al., 2011] Bo Xie, Yang Mu, Dacheng Tao, and Kaiqi Huang. m-sne: Multiview stochastic neighbor embed- ding. IEEE Transactions on Systems, Man, and Cyber- netics, Part B (Cybernetics), 41(4):1088-1096, 2011.","title":"m-sne: Multiview stochastic neighbor embedding","authors":"Bo Xie, Yang Mu, Dacheng Tao, Kaiqi Huang","year":"2011","venue":"IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yi et al., 2012] Jinfeng Yi, Rong Jin, Shaili Jain, Tianbao Yang, and Anil K. Jain. Semi-crowdsourced clustering: Generalizing crowd labeling by robust distance metric learning. In Advances in Neural Information Processing Systems 25, pages 1772-1780. 2012.","title":"Semi-crowdsourced clustering: Generalizing crowd labeling by robust distance metric learning","authors":"Jinfeng Yi, Rong Jin, Shaili Jain, Tianbao Yang, Anil K Jain","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}