{"type":"ScholarlyArticle","title":"Learning with Labeled and Unlabeled Multi-Step Transition Data for Recovering Markov Chain from Incomplete Transition Data","authors":[{"name":"Masahiro Kohjima","slug":"masahiro-kohjima","given":"Masahiro","family":"Kohjima","affiliation":"NTT Service Evolution Laboratories , NTT Corporation , Japan"},{"name":"Takeshi Kurashima","slug":"takeshi-kurashima","given":"Takeshi","family":"Kurashima","affiliation":"NTT Service Evolution Laboratories , NTT Corporation , Japan"},{"name":"Hiroyuki Toda","slug":"hiroyuki-toda","given":"Hiroyuki","family":"Toda","affiliation":"NTT Service Evolution Laboratories , NTT Corporation , Japan"}],"abstract":"Due to the difficulty of comprehensive data collection, created by factors such as privacy protection and sensor device limitations, we often need to analyze incomplete transition data where some information is missing from the ideal (complete) transition data. In this paper, we propose a new method that can estimate, in a unified manner, Markov chain parameters from incomplete transition data that consist of hidden transition data (data from which visited state information is partially hidden) and dropped transition data (data from which some state visits are dropped). A key to developing the method is regarding the hidden and dropped transition data as labeled and unlabeled multi-step transition data, where the labels represent the number of steps required for each transition. This allows us to describe the generative process of multi-step transition data, and thus develop a new probabilistic model. We confirm the effectiveness of the proposal by experiments on synthetic and real data.","doi":"10.24963/ijcai.2020/334","url":"https://ijcai.org/proceedings/2020/334","pdf":"https://ijcai.org/proceedings/2020/0334.pdf","markdown":"https://ijcai.org/proceedings/2020/334.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/334","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":334,"pages":{"first":2412,"last":2419},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Learning Generative Models","Machine Learning: Semi-Supervised Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24659/file_24659.m3u8","length_sec":295,"provider":"hls"}],"references":[{"position":1,"raw":"[Akagi et al., 2018] Yasunori Akagi, Takuya Nishimura, Takeshi Kurashima, and Hiroyuki Toda. A fast and ac- curate method for estimating people flow from spatiotem- poral population data. In International Joint Conferences on Artificial Intelligence, pages 3293-3300, 2018.","title":"A fast and accurate method for estimating people flow from spatiotemporal population data","authors":"Yasunori Akagi, Takuya Nishimura, Takeshi Kurashima, Hiroyuki Toda","year":"2018","venue":"International Joint Conferences on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/457"},{"position":2,"raw":"[Amari and Nagaoka, 2007] Shun-ichi Amari and Hiroshi Nagaoka. Methods of information geometry, volume 191. American Mathematical Soc., 2007.","title":"Shun-ichi Amari and Hiroshi Nagaoka","authors":"Nagaoka Amari","year":"2007","venue":"Methods of information geometry","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Antoniak, 1974] Charles E Antoniak. Mixtures of dirich- let processes with applications to bayesian nonparamet- ric problems. The annals of statistics, pages 1152-1174, 1974.","title":"Mixtures of dirichlet processes with applications to bayesian nonparametric problems","authors":"Charles E Antoniak","year":"1974","venue":"The annals of statistics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Basu et al., 2002] Sugato Basu, Arindam Banerjee, and Raymond Mooney. Semi-supervised clustering by seed- ing. In In Proceedings of 19th International Conference on Machine Learning, 2002.","title":"Semi-supervised clustering by seeding","authors":"Sugato Basu, Arindam Banerjee, Raymond Mooney","year":"2002","venue":"Proceedings of 19th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Billingsley, 1961] Patrick Billingsley. Statistical inference for Markov processes. University of Chicago Press, 1961.","title":"Statistical inference for Markov processes","authors":"Patrick Billingsley","year":"1961","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Blei and Jordan, 2006] David M Blei and Michael I Jor- dan. Variational inference for dirichlet process mixtures. Bayesian analysis, 1(1):121-143, 2006.","title":"Variational inference for dirichlet process mixtures","authors":"M David, Michael I Blei, Jordan","year":"2006","venue":"Bayesian analysis","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Crisostomi et al., 2011] Emanuele Crisostomi, Stephen Kirkland, and Robert Shorten. A google-like model of road network dynamics and its application to reg- ulation and control. International Journal of Control, 84(3):633-651, 2011.","title":"A google-like model of road network dynamics and its application to regulation and control","authors":"Emanuele Crisostomi, Stephen Kirkland, Robert Shorten","year":"2011","venue":"International Journal of Control","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[De Leeuw, 1994] Jan De Leeuw. Block-relaxation algo- rithms in statistics. In Information systems and data anal- ysis, pages 308-324. Springer, 1994.","title":"Jan De Leeuw. Block-relaxation algorithms in statistics","authors":"De Leeuw","year":"1994","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dempster et al., 1977] Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incom- plete data via the em algorithm. Journal of the Royal Sta- tistical Society: Series B (Methodological), 39(1):1-22, 1977.","title":"Maximum likelihood from incomplete data via the em algorithm","authors":"Nan M Arthur P Dempster, Donald B Laird, Rubin","year":"1977","venue":"Journal of the Royal Statistical Society: Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Durrett, 1999] Richard Durrett. Essentials of stochastic pro- cesses, volume 1. Springer, 1999.","title":"Essentials of stochastic processes","authors":"Richard Durrett","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fan et al., 2015] Zipei Fan, Xuan Song, Ryosuke Shibasaki, and Ryutaro Adachi. Citymomentum: an online approach for crowd behavior prediction at a citywide level. In In- ternational Joint Conference on Pervasive and Ubiquitous Computing, pages 559-569, 2015.","title":"Citymomentum: an online approach for crowd behavior prediction at a citywide level","authors":"Fan","year":"2015","venue":"ternational Joint Conference on Pervasive and Ubiquitous Computing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ferguson, 1973] Thomas S Ferguson. A bayesian analysis of some nonparametric problems. The annals of statistics, pages 209-230, 1973.","title":"A bayesian analysis of some nonparametric problems","authors":"Thomas S Ferguson","year":"1973","venue":"The annals of statistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Frydman and Schuermann, 2008] Halina Frydman and Til Schuermann. Credit rating dynamics and markov mixture models. Journal of Banking & Finance, 32(6):1062-1075, 2008.","title":"Credit rating dynamics and markov mixture models","authors":"Halina Frydman, Til Schuermann","year":"2008","venue":"Journal of Banking & Finance","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Frydman, 1984] Halina Frydman. Maximum likelihood es- timation in the mover-stayer model. Journal of the Ameri- can Statistical Association, 79(387):632-638, 1984.","title":"Maximum likelihood estimation in the mover-stayer model","authors":"Halina Frydman","year":"1984","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ghahramani and Jordan, 1994] Zoubin Ghahramani and Michael I Jordan. Supervised learning from incom- plete data via an em approach. In Advances in neural information processing systems, pages 120-127, 1994.","title":"Supervised learning from incomplete data via an em approach","authors":"Jordan ; Ghahramani, Zoubin Ghahramani, Michael I Jordan","year":"1994","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Goodman, 1961] Leo A Goodman. Statistical methods for the mover-stayer model. Journal of the American Statisti- cal Association, 56(296):841-868, 1961.","title":"Statistical methods for the mover-stayer model","authors":"Leo A Goodman","year":"1961","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gupta et al., 2016] Rishi Gupta, Ravi Kumar, and Sergei Vassilvitskii. On mixtures of markov chains. In Advances in neural information processing systems, pages 3441- 3449, 2016.","title":"On mixtures of markov chains","authors":"Rishi Gupta, Ravi Kumar, Sergei Vassilvitskii","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hunter and Lange, 2004] David R Hunter and Kenneth Lange. A tutorial on mm algorithms. The American Statis- tician, 58(1):30-37, 2004.","title":"A tutorial on mm algorithms","authors":"R David, Kenneth Hunter, Lange","year":"2004","venue":"The American Statistician","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Iwata et al., 2017] Tomoharu Iwata, Hitoshi Shimizu, Fu- toshi Naya, and Naonori Ueda. Estimating people flow from spatiotemporal population data via collective graph- ical mixture models. ACM Transactions on Spatial Algo- rithms and Systems, 3(1):2, 2017.","title":"Estimating people flow from spatiotemporal population data via collective graphical mixture models","authors":"Tomoharu Iwata, Hitoshi Shimizu, Futoshi Naya, Naonori Ueda","year":"2017","venue":"ACM Transactions on Spatial Algorithms and Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kumar et al., 2015] Ravi Kumar, Andrew Tomkins, Sergei Vassilvitskii, and Erik Vee. Inverting a steady-state. In In- ternational Conference on Web Search and Data Mining, pages 359-368, 2015.","title":"Inverting a steady-state","authors":"Ravi Kumar, Andrew Tomkins, Sergei Vassilvitskii, Erik Vee","year":"2015","venue":"ternational Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Levin and Peres, 2017] David A Levin and Yuval Peres. Markov chains and mixing times. American Mathemati- cal Soc., 2017.","title":"Markov chains and mixing times","authors":"A David, Yuval Levin, Peres","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu and Nocedal, 1989] Dong C Liu and Jorge Nocedal. On the limited memory bfgs method for large scale optimiza- tion. Mathematical programming, 45(1-3):503-528, 1989.","title":"On the limited memory bfgs method for large scale optimization","authors":"C Dong, Jorge Liu, Nocedal","year":"1989","venue":"Mathematical programming","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Morimura et al., 2009] Tetsuro Morimura, Eiji Uchibe, Ju- nichiro Yoshimoto, and Kenji Doya. A generalized natural actor-critic algorithm. In Advances in Neural Information Processing Systems, pages 1312-1320, 2009.","title":"A generalized natural actor-critic algorithm","authors":"Tetsuro Morimura, Eiji Uchibe, Junichiro Yoshimoto, Kenji Doya","year":"2009","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Morimura et al., 2013] Tetsuro Morimura, Takayuki Os- ogami, and Tsuyoshi Idé. Solving inverse problem of markov chain with partial observations. In Advances in Neural Information Processing Systems, pages 1655- 1663, 2013.","title":"Solving inverse problem of markov chain with partial observations","authors":"Tetsuro Morimura, Takayuki Osogami, Tsuyoshi Idé","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Neal, 2000] Radford M Neal. Markov chain sampling meth- ods for dirichlet process mixture models. Journal of com- putational and graphical statistics, 9(2):249-265, 2000.","title":"Markov chain sampling methods for dirichlet process mixture models","authors":"M Radford, Neal","year":"2000","venue":"Journal of computational and graphical statistics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Neuts, 1981] Marcel F Neuts. Matrix-geometric solutions in stochastic models: an algorithmic approach. Johns Hopkins University Press, 1981.","title":"Matrix-geometric solutions in stochastic models: an algorithmic approach","authors":"F Marcel, Neuts","year":"1981","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Nigam et al., 2000] Kamal Nigam, Andrew Kachites Mc- Callum, Sebastian Thrun, and Tom Mitchell. Text classi- fication from labeled and unlabeled documents using em. Machine learning, 39(2-3):103-134, 2000.","title":"Text classification from labeled and unlabeled documents using em","authors":"Kamal Nigam, Andrew Kachites Mc-Callum, Sebastian Thrun, Tom Mitchell","year":"2000","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Pfeifer and Carraway, 2000] Phillip E Pfeifer and Robert L Carraway. Modeling customer relationships as markov chains. Journal of interactive marketing, 14(2):43-55, 2000.","title":"Modeling customer relationships as markov chains","authors":"E Phillip, Robert L Pfeifer, Carraway","year":"2000","venue":"Journal of interactive marketing","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Resnick, 2002] Sidney I Resnick. Adventures in stochastic processes. Springer Science & Business Media, 2002.","title":"Adventures in stochastic processes","authors":"Sidney I Resnick","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Teh, 2010] Yee Whye Teh. Dirichlet process. Encyclopedia of machine learning, pages 280-287, 2010.","title":"Dirichlet process","authors":"Yee Whye Teh","year":"2010","venue":"Encyclopedia of machine learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}