{"type":"ScholarlyArticle","title":"Minimally Supervised Contextual Inference from Human Mobility: An Iterative Collaborative Distillation Framework","authors":[{"name":"Jiayun Zhang","slug":"jiayun-zhang","given":"Jiayun","family":"Zhang","affiliation":"University of California , San Diego"},{"name":"Xinyang Zhang","slug":"xinyang-zhang","given":"Xinyang","family":"Zhang","affiliation":"University of Illinois at Urbana-Champaign"},{"name":"Dezhi Hong","slug":"dezhi-hong","given":"Dezhi","family":"Hong","affiliation":"Amazon"},{"name":"Rajesh K. Gupta","slug":"rajesh-k-gupta","given":"Rajesh K.","family":"Gupta","affiliation":"University of California , San Diego"},{"name":"Jingbo Shang","slug":"jingbo-shang","given":"Jingbo","family":"Shang","affiliation":"University of California , San Diego"}],"abstract":"The context about trips and users from mobility data is valuable for mobile service providers to understand their customers and improve their services. Existing inference methods require a large number of labels for training, which is hard to meet in practice. In this paper, we study a more practical yet challenging setting—contextual inference using mobility data with minimal supervision (i.e., a few labels per class and massive unlabeled data). A typical solution is to apply semi-supervised methods that follow a self-training framework to bootstrap a model based on all features. However, using a limited labeled set brings high risk of overfitting to self-training, leading to unsatisfactory performance. We propose a novel collaborative distillation framework STCOLAB. It sequentially trains spatial and temporal modules at each iteration following the supervision of ground-truth labels. In addition, it distills knowledge to the module being trained using the logits produced by the latest trained module of the other modality, thereby mutually calibrating the two modules and combining the knowledge from both modalities. Extensive experiments on two real-world datasets show STCOLAB achieves significantly more accurate contextual inference than various baselines.","doi":"10.24963/ijcai.2023/272","url":"https://ijcai.org/proceedings/2023/272","pdf":"https://ijcai.org/proceedings/2023/0272.pdf","markdown":"https://ijcai.org/proceedings/2023/272.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/272","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Data Mining","track":"Main Track","number":272,"pages":{"first":2450,"last":2458},"keywords":["Data Mining: DM: Mining spatial and/or temporal data","Machine Learning: ML: Semi-supervised learning"],"videos":[],"references":[{"position":1,"raw":"[Arazo et al., 2020] Eric Arazo, Diego Ortego, Paul Albert, Noel E O'Connor, and Kevin McGuinness. Pseudo- Labeling and Confirmation Bias in Deep Semi-Supervised Learning. In 2020 International Joint Conference on Neu- ral Networks, pages 1-8, 2020.","title":"Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning","authors":"Eric Arazo, Diego Ortego, Paul Albert, E O' Noel, Kevin Connor, Mcguinness","year":"2020","venue":"2020 International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bi et al., 2013] Bin Bi, Milad Shokouhi, Michal Kosinski, and Thore Graepel. Inferring the Demographics of Search Users: Social Data Meets Search Queries. In Proceedings of the 22nd International Conference on World Wide Web, pages 131-140, 2013.","title":"Inferring the Demographics of Search Users: Social Data Meets Search Queries","authors":"Bi","year":"2013","venue":"Proceedings of the 22nd International Conference on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cascante-Bonilla et al., 2021] Paola Cascante-Bonilla, Fuwen Tan, Yanjun Qi, and Vicente Ordonez. Cur- riculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 6912-6920, 2021.","title":"Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning","authors":"Paola Cascante-Bonilla, Fuwen Tan, Yanjun Qi, Vicente Ordonez","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Culotta et al., 2015] Aron Culotta, Nirmal Kumar, and Jen- nifer Cutler. Predicting the Demographics of Twitter Users from Website Traffic Data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.","title":"Predicting the Demographics of Twitter Users from Website Traffic Data","authors":"Aron Culotta, Nirmal Kumar, Jennifer Cutler","year":"2015","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fang et al., 2021] Zhihan Fang, Yu Yang, Guang Yang, Yikuan Xian, Fan Zhang, and Desheng Zhang. CellSense: 4 https://firstdonoharm.dev Human Mobility Recovery via Cellular Network Data En- hancement. Proceedings of the ACM on Interactive, Mo- bile, Wearable and Ubiquitous Technologies, 5(3):1-22, 2021.","title":"CellSense: 4 https://firstdonoharm.dev Human Mobility Recovery via Cellular Network Data Enhancement","authors":"Zhihan Fang, Yu Yang, Guang Yang, Yikuan Xian, Fan Zhang, Desheng Zhang","year":"2021","venue":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-Agnostic Meta-Learning for Fast Adapta- tion of Deep Networks. In ICML, pages 1126-1135, 2017.","title":"Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks","authors":"Finn","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gat et al., 2020] Itai Gat, Idan Schwartz, Alexander Schwing, and Tamir Hazan. Removing Bias in Multi- modal Classifiers: Regularization by Maximizing Functional Entropies. Advances in Neural Information Processing Systems, 33:3197-3208, 2020.","title":"Removing Bias in Multimodal Classifiers: Regularization by Maximizing Functional Entropies","authors":"Itai Gat, Idan Schwartz, Alexander Schwing, Tamir Hazan","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the Knowledge in a Neural Network. arXiv preprint arXiv:1503.02531, 2(7), 2015.","title":"Distilling the Knowledge in a Neural Network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jordahl et al., 2020] Kelsey Jordahl, Joris Van den Bossche, Martin Fleischmann, Jacob Wasserman, James McBride, Jeffrey Gerard, Jeff Tratner, Matthew Perry, Adrian Garcia Badaracco, Carson Farmer, et al. geopan- das/geopandas: v0. 8.1. Zenodo, 2020.","title":"geopandas/geopandas","authors":"Kelsey Jordahl, Joris Van Den Bossche, Martin Fleischmann, Jacob Wasserman, James Mcbride, Jeffrey Gerard, Jeff Tratner, Matthew Perry, Adrian Garcia Badaracco, Carson Farmer","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim et al., 2019] Raehyun Kim, Hyunjae Kim, Janghyuk Lee, and Jaewoo Kang. Predicting Multiple Demographic Attributes with Task Specific Embedding Transformation and Attention Network. In Proceedings of the 2019 SIAM International Conference on Data Mining, pages 765-773, 2019.","title":"Predicting Multiple Demographic Attributes with Task Specific Embedding Transformation and Attention Network","authors":"Kim","year":"2019","venue":"Proceedings of the 2019 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Koch et al., 2015] Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. Siamese Neural Networks For One- Shot Image Recognition. In ICML deep learning work- shop, volume 2, 2015.","title":"Siamese Neural Networks For One-Shot Image Recognition","authors":"Gregory Koch, Richard Zemel, Ruslan Salakhutdinov","year":"2015","venue":"ICML deep learning workshop","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lee and others, 2013] Dong-Hyun Lee et al. Pseudo- Label: The Simple and Efficient Semi-Supervised Learn- ing Method for Deep Neural Networks. In Workshop on Challenges in Representation Learning, ICML, volume 3, page 896, 2013.","title":"Pseudo-Label: The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks","authors":"Dong-Hyun Lee","year":"2013","venue":"Workshop on Challenges in Representation Learning, ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2019] Xinzhe Li, Qianru Sun, Yaoyao Liu, Qin Zhou, Shibao Zheng, Tat-Seng Chua, and Bernt Schiele. Learning to Self-Train for Semi-Supervised Few-Shot Classification. Advances in Neural Information Process- ing Systems, 32:10276-10286, 2019.","title":"Learning to Self-Train for Semi-Supervised Few-Shot Classification","authors":"Li","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2017] Yanchi Liu, Chuanren Liu, Xinjiang Lu, Mingfei Teng, Hengshu Zhu, and Hui Xiong. Point-of- Interest Demand Modeling with Human Mobility Patterns. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 947-955, 2017.","title":"Point-of-Interest Demand Modeling with Human Mobility Patterns","authors":"Yanchi Liu, Chuanren Liu, Xinjiang Lu, Mingfei Teng, Hengshu Zhu, Hui Xiong","year":"2017","venue":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Luo et al., 2016] Feixiong Luo, Guofeng Cao, Kevin Mul- ligan, and Xiang Li. Explore spatiotemporal and demo- graphic characteristics of human mobility via Twitter: A case study of Chicago. Applied Geography, 70:11-25, 2016.","title":"Explore spatiotemporal and demographic characteristics of human mobility via Twitter: A case study of Chicago","authors":"Luo","year":"2016","venue":"Applied Geography","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Meng et al., 2017] Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, and Yu Chi. Deep Keyphrase Generation. arXiv preprint arXiv:1704.06879, 2017.","title":"Deep Keyphrase Generation","authors":"Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, Yu Chi","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Montasser and Kifer, 2017] Omar Montasser and Daniel Kifer. Predicting Demographics of High-Resolution Ge- ographies with Geotagged Tweets. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.","title":"Predicting Demographics of High-Resolution Geographies with Geotagged Tweets","authors":"Kifer Montasser, Omar Montasser, Daniel Kifer","year":"2017","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Quercia et al., 2010] Daniele Quercia, Neal Lathia, Francesco Calabrese, Giusy Di Lorenzo, and Jon Crowcroft. Recommending Social Events from Mobile Phone Location Data. In 2010 IEEE International Conference on Data Mining, pages 971-976, 2010.","title":"Recommending Social Events from Mobile Phone Location Data","authors":"Daniele Quercia, Neal Lathia, Francesco Calabrese, Giusy Di Lorenzo, Jon Crowcroft","year":"2010","venue":"2010 IEEE International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Schein et al., 2002] Andrew I Schein, Alexandrin Popescul, Lyle H Ungar, and David M Pennock. Methods and Met- rics for Cold-Start Recommendations. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 253-260, 2002.","title":"Methods and Metrics for Cold-Start Recommendations","authors":"Alexandrin Andrew I Schein, Lyle H Popescul, David M Ungar, Pennock","year":"2002","venue":"Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shang et al., 2018] Jin Shang, Mingxuan Sun, and Kevyn Collins-Thompson. Demographic Inference Via Knowl- edge Transfer in Cross-Domain Recommender Systems. In 2018 IEEE International Conference on Data Mining, pages 1218-1223, 2018.","title":"Demographic Inference Via Knowledge Transfer in Cross-Domain Recommender Systems","authors":"Jin Shang, Mingxuan Sun, Kevyn Collins-Thompson","year":"2018","venue":"2018 IEEE International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shen et al., 2018] Ting Shen, Haiquan Chen, and Wei- Shinn Ku. Time-aware Location Sequence Recommenda- tion for Cold-start Mobile Users. In Proceedings of 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pages 484-487, 2018.","title":"Time-aware Location Sequence Recommendation for Cold-start Mobile Users","authors":"Ting Shen, Haiquan Chen, Wei-Shinn Ku","year":"2018","venue":"Proceedings of 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shi et al., 2015] Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. Convolutional LSTM Network: A Machine Learning Ap- proach for Precipitation Nowcasting. Advances in Neural Information Processing Systems, 28, 2015.","title":"Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting","authors":"Shi","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shi et al., 2018] Weiwei Shi, Yihong Gong, Chris Ding, Zhiheng MaXiaoyu Tao, and Nanning Zheng. Transduc- tive Semi-Supervised Deep Learning using Min-Max Fea- tures. In Proceedings of the European Conference on Computer Vision, pages 299-315, 2018.","title":"Transductive Semi-Supervised Deep Learning using Min-Max Features","authors":"Weiwei Shi, Yihong Gong, Chris Ding, Zhiheng Maxiaoyu Tao, Nanning Zheng","year":"2018","venue":"Proceedings of the European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention Is All You Need. In Advances in Neural Information Processing Sys- tems, pages 5998-6008, 2017.","title":"Attention Is All You Need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2016] Pengfei Wang, Jiafeng Guo, Yanyan Lan, Jun Xu, and Xueqi Cheng. Your Cart tells You: In- ferring Demographic Attributes from Purchase Data. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, pages 173-182, 2016.","title":"Your Cart tells You: Inferring Demographic Attributes from Purchase Data","authors":"Pengfei Wang, Jiafeng Guo, Yanyan Lan, Jun Xu, Xueqi Cheng","year":"2016","venue":"Proceedings of the Ninth ACM International Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2017] Pinghui Wang, Feiyang Sun, Di Wang, Jing Tao, Xiaohong Guan, and Albert Bifet. Inferring De- mographics and Social Networks of Mobile Device Users on Campus From AP-Trajectories. In Proceedings of the 26th International Conference on World Wide Web Com- panion, pages 139-147, 2017.","title":"Inferring Demographics and Social Networks of Mobile Device Users on Campus From AP-Trajectories","authors":"Wang","year":"2017","venue":"Proceedings of the 26th International Conference on World Wide Web Companion","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2019] Zijian Wang, Scott Hale, David Ifeoluwa Adelani, Przemyslaw Grabowicz, Timo Hartman, Fabian Flöck, and David Jurgens. Demographic Inference and Representative Population Estimates from Multilingual Social Media Data. In Proceedings of the 2019 World Wide Web Conference, pages 2056-2067, 2019.","title":"Demographic Inference and Representative Population Estimates from Multilingual Social Media Data","authors":"Zijian Wang, Scott Hale, David Ifeoluwa Adelani, Przemyslaw Grabowicz, Timo Hartman, Fabian Flöck, David Jurgens","year":"2019","venue":"Proceedings of the 2019 World Wide Web Conference","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2020a] Daheng Wang, Meng Jiang, Munira Syed, Oliver Conway, Vishal Juneja, Sriram Subramanian, and Nitesh V. Chawla. Calendar graph neural networks for modeling time structures in spatiotemporal user be- haviors. In Proceedings of the 26th ACM SIGKDD In- ternational Conference on Knowledge Discovery & Data Mining, pages 2581-2589, 2020.","title":"Calendar graph neural networks for modeling time structures in spatiotemporal user behaviors","authors":"Wang","year":"2020","venue":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2020b] Qinyong Wang, Hongzhi Yin, Tong Chen, Zi Huang, Hao Wang, Yanchang Zhao, and Nguyen Quoc Viet Hung. Next Point-of-Interest Recom- mendation on Resource-Constrained Mobile Devices. In Proceedings of the Web Conference, pages 906-916, 2020.","title":"Next Point-of-Interest Recommendation on Resource-Constrained Mobile Devices","authors":"Wang","year":"2020","venue":"Proceedings of the Web Conference","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xu et al., 2020] Fengli Xu, Zongyu Lin, Tong Xia, Dian- sheng Guo, and Yong Li. SUME: Semantic-enhanced Ur- ban Mobility Network Embedding for User Demographic Inference. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(3):1-25, 2020.","title":"SUME: Semantic-enhanced Urban Mobility Network Embedding for User Demographic Inference","authors":"Fengli Xu, Zongyu Lin, Tong Xia, Diansheng Guo, Yong Li","year":"2020","venue":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yuan et al., 2018] Xingdi Yuan, Tong Wang, Rui Meng, Khushboo Thaker, Peter Brusilovsky, Daqing He, and Adam Trischler. One Size Does Not Fit All: Generat- ing and Evaluating Variable Number of Keyphrases. arXiv preprint arXiv:1810.05241, 2018.","title":"One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases","authors":"Xingdi Yuan, Tong Wang, Rui Meng, Khushboo Thaker, Peter Brusilovsky, Daqing He, Adam Trischler","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhang et al., 2016] Ke Zhang, Yu-Ru Lin, and Konstanti- nos Pelechrinis. EigenTransitions with Hypothesis Test- ing: The Anatomy of Urban Mobility. In Proceedings of the International AAAI Conference on Web and Social Me- dia, volume 10, 2016.","title":"EigenTransitions with Hypothesis Testing: The Anatomy of Urban Mobility","authors":"Ke Zhang, Yu-Ru Lin, Konstantinos Pelechrinis","year":"2016","venue":"Proceedings of the International AAAI Conference on Web and Social Media","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2020] Yu Zhang, Yu Meng, Jiaxin Huang, Frank F Xu, Xuan Wang, and Jiawei Han. Minimally Su- pervised Categorization of Text with Metadata. In Pro- ceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1231-1240, 2020.","title":"Minimally Supervised Categorization of Text with Metadata","authors":"Yu Zhang, Yu Meng, Jiaxin Huang, Frank F Xu, Xuan Wang, Jiawei Han","year":"2020","venue":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang et al., 2021] Xinyang Zhang, Chenwei Zhang, Xin Luna Dong, Jingbo Shang, and Jiawei Han. Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks. In Proceedings of the Web Conference, pages 3258-3268, 2021.","title":"Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks","authors":"Xinyang Zhang, Chenwei Zhang, Xin, Luna Dong, Jingbo Shang, Jiawei Han","year":"2021","venue":"Proceedings of the Web Conference","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhong et al., 2015] Yuan Zhong, Nicholas Jing Yuan, Wen Zhong, Fuzheng Zhang, and Xing Xie. You Are Where You Go: Inferring Demographic Attributes from Location Check-ins. In Proceedings of the Eighth ACM Interna- tional Conference on Web Search and Data Mining, pages 295-304, 2015.","title":"You Are Where You Go: Inferring Demographic Attributes from Location Check-ins","authors":"Yuan Zhong, Nicholas Jing Yuan, Wen Zhong, Fuzheng Zhang, Xing Xie","year":"2015","venue":"Proceedings of the Eighth ACM International Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}