{"type":"ScholarlyArticle","title":"Prediction with Incomplete Data under Agnostic Mask Distribution Shift","authors":[{"name":"Yichen Zhu","slug":"yichen-zhu","given":"Yichen","family":"Zhu","affiliation":"Shanghai Jiao Tong University"},{"name":"Jian Yuan","slug":"jian-yuan","given":"Jian","family":"Yuan","affiliation":"Shanghai Jiao Tong University"},{"name":"Bo Jiang","slug":"bo-jiang","given":"Bo","family":"Jiang","affiliation":"Shanghai Jiao Tong University"},{"name":"Tao Lin","slug":"tao-lin","given":"Tao","family":"Lin","affiliation":"Communication University of China {zyc"},{"name":"Haiming Jin","slug":"haiming-jin","given":"Haiming","family":"Jin","affiliation":"Shanghai Jiao Tong University"},{"name":"Xinbing Wang","slug":"xinbing-wang","given":"Xinbing","family":"Wang","affiliation":"Shanghai Jiao Tong University"},{"name":"Chenghu Zhou","slug":"chenghu-zhou","given":"Chenghu","family":"Zhou","affiliation":"Shanghai Jiao Tong University"}],"abstract":"Data with missing values is ubiquitous in many applications. Recent years have witnessed increasing attention on prediction with only incomplete data consisting of observed features and a mask that indicates the missing pattern. Existing methods assume that the training and testing distributions are the same, which may be violated in real-world scenarios. In this paper, we consider prediction with incomplete data in the presence of distribution shift. We focus on the case where the underlying joint distribution of complete features and label is invariant, but the missing pattern, i.e., mask distribution may shift agnostically between training and testing. To achieve generalization, we leverage the observation that for each mask, there is an invariant optimal predictor. To avoid the exponential explosion when learning them separately, we approximate the optimal predictors jointly using a double parameterization technique. This has the undesirable side effect of allowing the learned predictors to rely on the intra-mask correlation and that between features and mask. We perform decorrelation to minimize this effect. Combining the techniques above, we propose a novel prediction method called StableMiss. Extensive experiments on both synthetic and real-world datasets show that StableMiss is robust and outperforms state-of-the-art methods under agnostic mask distribution shift.","doi":"10.24963/ijcai.2023/525","url":"https://ijcai.org/proceedings/2023/525","pdf":"https://ijcai.org/proceedings/2023/0525.pdf","markdown":"https://ijcai.org/proceedings/2023/525.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/525","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":525,"pages":{"first":4720,"last":4728},"keywords":["Machine Learning: ML: Multi-task and transfer learning"],"videos":[],"references":[{"position":1,"raw":"[Burda et al., 2016] Yuri Burda, Roger B. Grosse, and Rus- lan Salakhutdinov. Importance weighted autoencoders. In The 4th International Conference on Learning Represen- tations, San Juan, Puerto Rico, 2016.","title":"Importance weighted autoencoders","authors":"Yuri Burda, Roger B Grosse, Ruslan Salakhutdinov","year":"2016","venue":"The 4th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[DiDiChuxing, 2018] DiDiChuxing. City traffic index. https://gaia.didichuxing.com/, 2018. Accessed: 2018-12- 31.","title":"DiDiChuxing. City traffic index","authors":"Didichuxing","year":"2018","venue":null,"doi":null,"url":"https://gaia.didichuxing.com/,2018","internal_url":null},{"position":3,"raw":"[Donahue et al., 2017] Jeff Donahue, Philipp Krahenbuhl, and Trevor Darrell. Adversarial feature learning. In The 5th International Conference on Learning Represen- tations, Toulon, France, 2017.","title":"Adversarial feature learning","authors":"Jeff Donahue, Philipp Krahenbuhl, Trevor Darrell","year":"2017","venue":"The 5th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances in Neural Infor- mation Processing Systems, pages 2672-2680, Montreal, Canada, December 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Kingma and Welling, 2014] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In The 2nd International Conference on Learning Representations, Banff, Canada, 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"The 2nd International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Kuang et al., 2018] Kun Kuang, Peng Cui, Susan Athey, Ruoxuan Xiong, and Bo Li. Stable prediction across un- known environments. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discov- ery & Data Mining, pages 1617-1626, London, UK, 2018.","title":"Stable prediction across unknown environments","authors":"Kun Kuang, Peng Cui, Susan Athey, Ruoxuan Xiong, Bo Li","year":"2018","venue":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kuang et al., 2020] Kun Kuang, Ruoxuan Xiong, Peng Cui, Susan Athey, and Bo Li. Stable prediction with model mis- specification and agnostic distribution shift. In The 34th AAAI Conference on Artificial Intelligence, pages 4485- 4492, New York, USA, 2020.","title":"Stable prediction with model misspecification and agnostic distribution shift","authors":"Kun Kuang, Ruoxuan Xiong, Peng Cui, Susan Athey, Bo Li","year":"2020","venue":"The 34th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lecun et al., 1998] Yann Lecun, Leon Bottou, Yoshua Ben- gio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Leon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li and Marlin, 2020] Steven Cheng-Xian Li and Benjamin Marlin. Learning from irregularly-sampled time series: A missing data perspective. In Proceedings of the 37th Inter- national Conference on Machine Learning, pages 5937- 5946, Virtual Event, 2020.","title":"Learning from irregularly-sampled time series: A missing data perspective","authors":"Steven Cheng-Xian, Li, Benjamin Marlin","year":"2020","venue":"Proceedings of the 37th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2019] Steven Cheng-Xian Li, Bo Jiang, and Ben- jamin Marlin. Learning from incomplete data with gener- ative adversarial networks. In Proceedings of the 7th In- ternational Conference on Learning Representations, New Orleans, USA, May 2019.","title":"Learning from incomplete data with generative adversarial networks","authors":"Steven Cheng-Xian, Bo Li, Benjamin Jiang, Marlin","year":"2019","venue":"Proceedings of the 7th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Little and Rubin, 1986] Roderick J. A. Little and Donald B. Rubin. Statistical Analysis with Missing Data. John Wiley & Sons, Inc., USA, 1986.","title":"Statistical Analysis with Missing Data","authors":"J A Roderick, Donald B Little, Rubin","year":"1986","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ma et al., 2019] Chao Ma, Sebastian Tschiatschek, Kon- stantina Palla, Jose Miguel Hernandez-Lobato, Sebastian Nowozin, and Cheng Zhang. EDDI: Efficient dynamic dis- covery of high-value information with partial vae. In Pro- ceedings of the 36th International Conference on Machine Learning, pages 4234-4243, Long Beach, USA, 2019.","title":"EDDI: Efficient dynamic discovery of high-value information with partial vae","authors":"Chao Ma, Sebastian Tschiatschek, Konstantina Palla, Jose, Miguel Hernandez-Lobato, Sebastian Nowozin, Cheng Zhang","year":"2019","venue":"Proceedings of the 36th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mattei and Frellsen, 2019] Pierre-Alexandre Mattei and Jes Frellsen. MIWAE: Deep generative modelling and impu- tation of incomplete data sets. In Proceedings of the 36th International Conference on Machine Learning, pages 4413-4423, Long Beach, USA, 2019.","title":"MIWAE: Deep generative modelling and imputation of incomplete data sets","authors":"Pierre-Alexandre Mattei, Jes Frellsen","year":"2019","venue":"Proceedings of the 36th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Morvan et al., 2020] Marine Le Morvan, Julie Josse, Thomas Moreau, Erwan Scornet, and Gael Varoquaux. NeuMiss networks: Differentiable programming for su- pervised learning with missing values. In Proceedings of the 34th International Conference on Neural Information Processing Systems, page 5980-5990, Vancouver, Canada, 2020.","title":"NeuMiss networks: Differentiable programming for supervised learning with missing values","authors":"Morvan","year":"2020","venue":"Proceedings of the 34th International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Morvan et al., 2021] Marine Le Morvan, Julie Josse, Erwan Scornet, and Gael Varoquaux. What's a good imputation to predict with mmissing values? In Proceedings of the 35th International Conference on Neural Information Process- ing Systems, pages 11530-11540, Virtual Event, 2021.","title":"What's a good imputation to predict with mmissing values?","authors":"Morvan","year":"2021","venue":"Proceedings of the 35th International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pan and Yang, 2010] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on Knowl- edge and Data Engineering, 22(10):1345-1359, 2010.","title":"A survey on transfer learning","authors":"Jialin Sinno, Qiang Pan, Yang","year":"2010","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shen et al., 2018] Zheyan Shen, Peng Cui, Kun Kuang, Bo Li, and Peixuan Chen. Causally regularized learning with agnostic data selection bias. In Proceedings of the 26th ACM International Conference on Multimedia, page 411-419, Seoul, Republic of Korea, 2018.","title":"Causally regularized learning with agnostic data selection bias","authors":"Zheyan Shen, Peng Cui, Kun Kuang, Bo Li, Peixuan Chen","year":"2018","venue":"Proceedings of the 26th ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shen et al., 2020] Zheyan Shen, Peng Cui, Tong Zhang, and Kun Kuang. Stable learning via sample reweighting. In The 34th AAAI Conference on Artificial Intelligence, pages 5692-5699, New York, USA, 2020.","title":"Stable learning via sample reweighting","authors":"Zheyan Shen, Peng Cui, Tong Zhang, Kun Kuang","year":"2020","venue":"The 34th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Strobl et al., 2019] Eric V. Strobl, Kun Zhang, and Shyam Visweswaran. Approximate kernel-based conditional in- dependence tests for fast non-parametric causal discovery. Journal of Causal Inference, 7(1), 2019.","title":"Approximate kernel-based conditional independence tests for fast non-parametric causal discovery","authors":"Eric V Strobl, Kun Zhang, Shyam Visweswaran","year":"2019","venue":"Journal of Causal Inference","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sugiyama et al., 2012] Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density Ratio Estimation in Ma- chine Learning. Cambridge University Press, UK, 2012.","title":"Density Ratio Estimation in Machine Learning","authors":"Sugiyama","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu et al., 2022] Renzhe Xu, Xingxuan Zhang, Zheyan Shen, Tong Zhang, and Peng Cui. A theoretical analysis on independence-driven importance weighting for covariate- shift generalization. In Proceedings of the 39th Interna- tional Conference on Machine Learning, pages 24803- 24829, Baltimore, USA, 2022.","title":"A theoretical analysis on independence-driven importance weighting for covariateshift generalization","authors":"Xu","year":"2022","venue":"Proceedings of the 39th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yoon et al., 2018] Jinsung Yoon, James Jordon, and Mi- haela van der Schaar. GAIN: Missing data imputation us- ing generative adversarial nets. In Proceedings of the 35th International Conference on Machine Learning, pages 5675-5684, Stockholmsmassan, Sweden, July 2018.","title":"GAIN: Missing data imputation using generative adversarial nets","authors":"Yoon","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2021] Xingxuan Zhang, Peng Cui, Renzhe Xu, Linjun Zhou, Yue He, and Zheyan Shen. Deep stable learning for out-of-distribution generalization. In IEEE Conference on Computer Vision and Pattern Recognition, pages 5372-5382, Virtual Event, 2021.","title":"Deep stable learning for out-of-distribution generalization","authors":"Xingxuan Zhang, Peng Cui, Renzhe Xu, Linjun Zhou, Yue He, Zheyan Shen","year":"2021","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}