{"type":"ScholarlyArticle","title":"Coarse-to-Fine Lifted MAP Inference in Computer Vision","authors":[{"name":"Haroun Habeeb","slug":"haroun-habeeb","given":"Haroun","family":"Habeeb","affiliation":"Indian Institute of Technology Delhi"},{"name":"Ankit Anand","slug":"ankit-anand","given":"Ankit","family":"Anand","affiliation":"Indian Institute of Technology Delhi"},{"name":"Mausam","slug":"mausam","given":"","family":"Mausam","affiliation":""},{"name":"Parag Singla","slug":"parag-singla","given":"Parag","family":"Singla","affiliation":"Indian Institute of Technology Delhi"}],"abstract":"There is a vast body of theoretical research on lifted inference in probabilistic graphical models (PGMs). However, few demonstrations exist where lifting is applied in conjunction with top of the line applied algorithms. We pursue the applicability of lifted inference for computer vision (CV), with the insight that a globally optimal (MAP) labeling will likely have the same label for two symmetric pixels. The success of our approach lies in efficiently handling a distinct unary potential on every node (pixel), typical of CV applications. This allows us to lift the large class of algorithms that model a CV problem via PGM inference. We propose a generic template for coarse-to-fine (C2F) inference in CV, which progressively refines an initial coarsely lifted PGM for varying quality-time trade-offs. We demonstrate the performance of C2F inference by developing lifted versions of two near state-of-the-art CV algorithms for stereo vision and interactive image segmentation. We find that, against flat algorithms, the lifted versions have a much superior anytime performance, without any loss in final solution quality.","doi":"10.24963/ijcai.2017/641","url":"https://ijcai.org/proceedings/2017/641","pdf":"https://ijcai.org/proceedings/2017/0641.pdf","markdown":"https://ijcai.org/proceedings/2017/641.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/641","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Uncertainty in AI","track":"Main track","number":641,"pages":{"first":4595,"last":4602},"keywords":["Uncertainty in AI: Approximate Probabilistic Inference","Uncertainty in AI: Graphical Models"],"videos":[],"references":[{"position":1,"raw":"[Achanta et al., 2012] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Ssstrunk. SLIC Superpixels Compared to State-of- the-Art Superpixel Methods. In PAMI, Nov 2012.","title":"SLIC Superpixels Compared to State-ofthe-Art Superpixel Methods","authors":"R Achanta, A Shaji, K Smith, A Lucchi, P Fua, S Ssstrunk","year":"2012","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Anand et al., 2016] A. Anand, A. Grover, Mausam, and P. Singla. Contextual Symmetries in Probabilistic Graphical Models. In IJ- CAI, 2016.","title":"Contextual Symmetries in Probabilistic Graphical Models","authors":"A Anand, A Grover, P Mausam, Singla","year":"2016","venue":"IJ-CAI","doi":null,"url":null,"internal_url":"/proceedings/2016/502"},{"position":3,"raw":"[Anand et al., 2017] A. Anand, R. Noothigattu, P. Singla, and Mausam. Non-Count Symmetries in Boolean & Multi-Valued Prob. Graphical Models. In AISTATS, 2017.","title":"Non-Count Symmetries in Boolean & Multi-Valued Prob. Graphical Models","authors":"A Anand, R Noothigattu, P Singla, Mausam","year":"2017","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Andres et al., 2010] B. Andres, J. H. Kappes, U. Köthe, C. Schnörr, and F. A. Hamprecht. An Empirical Comparison of Inference Algorithms for Graphical Models with Higher Order Factors Using OpenGM. In Pattern Recognition. 2010.","title":"An Empirical Comparison of Inference Algorithms for Graphical Models with Higher Order Factors Using OpenGM","authors":"B Andres, J H Kappes, U Köthe, C Schnörr, F A Hamprecht","year":"2010","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Blei et al., 2003] D. Blei, A. Ng, and M. Jordan. Latent Dirichlet Allocation. JMLR, 3, March 2003.","title":"Latent Dirichlet Allocation","authors":"D Blei, A Ng, M Jordan","year":"2003","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Boykov et al., 2001] Y. Boykov, O. Veksler, and R. Zabih. Fast Approximate Energy Minimization via Graph Cuts. In PAMI, 23(11), November 2001.","title":"Fast Approximate Energy Minimization via Graph Cuts","authors":"Y Boykov, O Veksler, R Zabih","year":"2001","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Braz et al., 2005] R. Braz, E. Amir, and D. Roth. Lifted First- Order Probabilistic Inference. In IJCAI, 2005.","title":"Lifted First-Order Probabilistic Inference","authors":"R Braz, E Amir, D Roth","year":"2005","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2005/204"},{"position":8,"raw":"[Bui et al., 2012] H. Bui, T. Huynh, and R. De Salvo Braz. Exact Lifted Inference with Distinct Soft Evidence on Every Object. In AAAI, 2012.","title":"Exact Lifted Inference with Distinct Soft Evidence on Every Object","authors":"H Bui, T Huynh, R De Salvo, Braz","year":"2012","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Bui et al., 2013] H. Bui, T. Huynh, and S. Riedel. Automorphism Groups of Graphical Models and Lifted Variational Inference. In UAI, 2013.","title":"Automorphism Groups of Graphical Models and Lifted Variational Inference","authors":"H Bui, T Huynh, S Riedel","year":"2013","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Freeman et al., 2000] W. Freeman, E. Pasztor, and O. Carmichael. Learning Low-Level Vision. In IJCV, 40, 2000.","title":"Learning Low-Level Vision","authors":"W Freeman, E Pasztor, O Carmichael","year":"2000","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Friedman, 2004] N. Friedman. Inferring Cellular Networks using Probabilistic Graphical Models. Science, 303, 2004.","title":"Inferring Cellular Networks using Probabilistic Graphical Models","authors":"N Friedman","year":"2004","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gogate and Domingos, 2011] V. Gogate and P. Domingos. Proba- bilisitic Theorem Proving. In UAI, 2011.","title":"Probabilisitic Theorem Proving","authors":"V Gogate, P Domingos","year":"2011","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hirschmuller and Scharstein, 2007] H. Hirschmuller and D. Scharstein. Evaluation of Cost Functions for Stereo Matching. In CVPR, 2007.","title":"Evaluation of Cost Functions for Stereo Matching","authors":"H Hirschmuller, D Scharstein","year":"2007","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jegelka and Bilmes, 2011] S. Jegelka and J. Bilmes. Submodu- larity Beyond Submodular Energies: Coupling Edges in Graph Cuts. In CVPR, 2011.","title":"Submodularity Beyond Submodular Energies: Coupling Edges in Graph Cuts","authors":"S Jegelka, J Bilmes","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jernite et al., 2015] Y. Jernite, A. Rush, and D. Sontag. A Fast Variational Approach for Learning Markov Random Field Lan- guage Models. In ICML, 2015.","title":"A Fast Variational Approach for Learning Markov Random Field Language Models","authors":"Y Jernite, A Rush, D Sontag","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jha et al., 2010] A. Jha, V. Gogate, A. Meliou, and D. Suciu. Lifted Inference Seen from the Other Side : The Tractable Fea- tures. In NIPS, 2010.","title":"Lifted Inference Seen from the Other Side : The Tractable Features","authors":"A Jha, V Gogate, A Meliou, D Suciu","year":"2010","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kappes et al., 2015] J. Kappes, B. Andres, A. Hamprecht, C. Schnörr, S. Nowozin, D. Batra, S. Kim, B. Kausler, T. Kröger, J. Lellmann, N. Komodakis, B. Savchynskyy, and C. Rother. A Comparative Study of Modern Inference Techniques for Struc- tured Discrete Energy Minimization Problems. In IJCV, 2015.","title":"A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems","authors":"J Kappes, B Andres, A Hamprecht, C Schnörr, S Nowozin, D Batra, S Kim, B Kausler, T Kröger, J Lellmann, N Komodakis, B Savchynskyy, C Rother","year":"2015","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kersting et al., 2009] K. Kersting, B. Ahmadi, and S. Natarajan. Counting Belief Propagation. In UAI, 2009.","title":"Counting Belief Propagation","authors":"K Kersting, B Ahmadi, S Natarajan","year":"2009","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kersting, 2012] K. Kersting. Lifted Probabilistic Inference. In ECAI, 2012.","title":"Lifted Probabilistic Inference","authors":"K Kersting","year":"2012","venue":"ECAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kiddon and Domingos, 2011] C. Kiddon and P. Domingos. Coarse-to-Fine Inference and Learning for First-Order Proba- bilistic Models. In AAAI, 2011.","title":"Coarse-to-Fine Inference and Learning for First-Order Probabilistic Models","authors":"C Kiddon, P Domingos","year":"2011","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kimmig et al., 2015] A. Kimmig, L. Mihalkova, and L. Getoor. Lifted Graphical Models: A Survey. Machine Learning, 2015.","title":"Lifted Graphical Models: A Survey","authors":"A Kimmig, L Mihalkova, L Getoor","year":"2015","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kohli et al., 2013] P. Kohli, A. Osokin, and S. Jegelka. A Prin- cipled Deep Random Field Model for Image Segmentation. In CVPR, 2013.","title":"A Principled Deep Random Field Model for Image Segmentation","authors":"P Kohli, A Osokin, S Jegelka","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lempitsky et al., 2010] V. Lempitsky, C. Rother, S. Roth, and A. Blake. Fusion Moves for Markov Random Field Optimiza- tion. In PAMI, Aug 2010.","title":"Fusion Moves for Markov Random Field Optimization","authors":"V Lempitsky, C Rother, S Roth, A Blake","year":"2010","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Mittal et al., 2014] H. Mittal, P. Goyal, V. Gogate, and P. Singla. New Rules for Domain Independent Lifted MAP Inference. In NIPS, 2014.","title":"New Rules for Domain Independent Lifted MAP Inference","authors":"H Mittal, P Goyal, V Gogate, P Singla","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mittal et al., 2015] H. Mittal, A. Mahajan, V. Gogate, and P. Singla. Lifted Inference Rules With Constraints. In NIPS, 2015.","title":"Lifted Inference Rules With Constraints","authors":"H Mittal, A Mahajan, V Gogate, P Singla","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Mladenov et al., 2014] M. Mladenov, K. Kersting, and A. Glober- son. Efficient Lifting of MAP LP Relaxations Using k-Locality. In AISTATS, 2014.","title":"Efficient Lifting of MAP LP Relaxations Using k-Locality","authors":"M Mladenov, K Kersting, A Globerson","year":"2014","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Mozerov and van de Weijer, 2015] M. G. Mozerov and J. van de Weijer. Accurate Stereo Matching by Two-Step Energy Mini- mization. IEEE Transactions on Image Processing, March 2015.","title":"Accurate Stereo Matching by Two-Step Energy Minimization","authors":"M G Mozerov, J Van De Weijer","year":"2015","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Nath and Domingos, 2010] A. Nath and P. Domingos. Efficient Lifting for Online Probabilistic Inference. In AAAIWS, 2010.","title":"Efficient Lifting for Online Probabilistic Inference","authors":"A Nath, P Domingos","year":"2010","venue":"AAAIWS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Nath and Domingos, 2016] A. Nath and P. Domingos. Learning Tractable Probabilistic Models for Fault Localization. In AAAI, 2016.","title":"Learning Tractable Probabilistic Models for Fault Localization","authors":"A Nath, P Domingos","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Niepert, 2012] M. Niepert. Markov Chains on Orbits of Permuta- tion Groups. In UAI, 2012.","title":"Markov Chains on Orbits of Permutation Groups","authors":"M Niepert","year":"2012","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Noessner et al., 2013] J. Noessner, M. Niepert, and H. Stucken- schmidt. RockIt: Exploiting Parallelism and Symmetry for MAP Inference in Statistical Relational Models. In AAAI, 2013.","title":"RockIt: Exploiting Parallelism and Symmetry for MAP Inference in Statistical Relational Models","authors":"J Noessner, M Niepert, H Stuckenschmidt","year":"2013","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Poole, 2003] D. Poole. First-Order Probabilistic Inference. In IJ- CAI, 2003.","title":"First-Order Probabilistic Inference","authors":"D Poole","year":"2003","venue":"IJ-CAI","doi":null,"url":null,"internal_url":"/proceedings/2003/141"},{"position":33,"raw":"[Sarkhel et al., 2014] S. Sarkhel, D. Venugopal, P. Singla, and V. Gogate. Lifted MAP inference for Markov logic networks. In AISTATS, 2014.","title":"Lifted MAP inference for Markov logic networks","authors":"S Sarkhel, D Venugopal, P Singla, V Gogate","year":"2014","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Sarkhel et al., 2015] S. Sarkhel, P. Singla, and V. Gogate. Fast Lifted MAP Inference via Partitioning. In NIPS, 2015.","title":"Fast Lifted MAP Inference via Partitioning","authors":"S Sarkhel, P Singla, V Gogate","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Scharstein and Szeliski, 2002] D. Scharstein and R. Szeliski. A Taxonomy and Evaluation of Dense Two-Frame Stereo Corre- spondence Algorithms. In IJCV, 2002.","title":"A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms","authors":"D Scharstein, R Szeliski","year":"2002","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Scharstein and Szeliski, 2003] D. Scharstein and R. Szeliski. High-accuracy Stereo Depth Maps Using Structured Light. In CVPR, 2003.","title":"High-accuracy Stereo Depth Maps Using Structured Light","authors":"D Scharstein, R Szeliski","year":"2003","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Singla and Domingos, 2008] P. Singla and P. Domingos. Lifted First-Order Belief Propagation. In AAAI, 2008.","title":"Lifted First-Order Belief Propagation","authors":"P Singla, P Domingos","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Singla et al., 2014] P. Singla, A. Nath, and P. Domingos. Approx- imate Lifting Techniques for Belief Propagation. In AAAI, 2014.","title":"Approximate Lifting Techniques for Belief Propagation","authors":"P Singla, A Nath, P Domingos","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Szeliski et al., 2008] R. Szeliski, R. Zabih, D. Scharstein, O. Vek- sler, V. Kolmogorov, A. Agarwala, M. Tappen, and C. Rother. A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors. In PAMI, June 2008.","title":"A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors","authors":"R Szeliski, R Zabih, D Scharstein, O Veksler, V Kolmogorov, A Agarwala, M Tappen, C Rother","year":"2008","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Van den Bergh et al., 2012] M. Van den Bergh, X. Boix, G. Roig, B. de Capitani, and L. Van Gool. SEEDS: Superpixels Extracted via Energy-Driven Sampling. In ECCV, 2012.","title":"SEEDS: Superpixels Extracted via Energy-Driven Sampling","authors":"M Van Den Bergh, X Boix, G Roig, B De Capitani, L Van Gool","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Van den Broeck and Darwiche, 2013] G. Van den Broeck and A. Darwiche. On the Complexity and Approximation of Binary Evidence in Lifted Inference. In NIPS, 2013.","title":"On the Complexity and Approximation of Binary Evidence in Lifted Inference","authors":"Broeck Van Den, ] G Darwiche, A Van Den Broeck, Darwiche","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Van den Broeck and Niepert, 2015] G. Van den Broeck and M. Niepert. Lifted Probabilistic Inference for Asymmetric Graphical Models. In AAAI, 2015.","title":"Lifted Probabilistic Inference for Asymmetric Graphical Models","authors":"G Van Den Broeck, M Niepert","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Venugopal and Gogate, 2014] D. Venugopal and V. Gogate. Evidence-Based Clustering for Scalable Inference in Markov Logic. In Joint ECML-KDD, 2014.","title":"Evidence-Based Clustering for Scalable Inference in Markov Logic","authors":"D Venugopal, V Gogate","year":"2014","venue":"Joint ECML-KDD","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wei et al., 2016] X. Wei, Q. Yang, Y. Gong, M. Yang, and N. Ahuja. Superpixel Hierarchy. CoRR, abs/1605.06325, 2016.","title":"Superpixel Hierarchy","authors":"Wei","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}