{"type":"ScholarlyArticle","title":"On Approximating Total Variation Distance","authors":[{"name":"Arnab Bhattacharyya","slug":"arnab-bhattacharyya","given":"Arnab","family":"Bhattacharyya","affiliation":"National University of Singapore"},{"name":"Sutanu Gayen","slug":"sutanu-gayen","given":"Sutanu","family":"Gayen","affiliation":"Indian Institute of Technology Kanpur"},{"name":"Kuldeep S. Meel","slug":"kuldeep-s-meel","given":"Kuldeep S.","family":"Meel","affiliation":"National University of Singapore"},{"name":"Dimitrios Myrisiotis","slug":"dimitrios-myrisiotis","given":"Dimitrios","family":"Myrisiotis","affiliation":"National University of Singapore"},{"name":"A. Pavan","slug":"a-pavan","given":"A.","family":"Pavan","affiliation":"Iowa State University"},{"name":"N. V. Vinodchandran","slug":"n-v-vinodchandran","given":"N. V.","family":"Vinodchandran","affiliation":"University of Nebraska-Lincoln"}],"abstract":"Total variation distance (TV distance) is a fundamental notion of distance between probability distributions. In this work, we introduce and study the problem of computing the TV distance of two product distributions over the domain {0,1}^n. In particular, we establish the following results.\r\n\r\n1. The problem of exactly computing the TV distance of two product distributions is #P-complete. This is in stark contrast with other distance measures such as KL, Chi-square, and Hellinger which tensorize over the marginals leading to efficient algorithms.\r\n\r\n2. There is a fully polynomial-time deterministic approximation scheme (FPTAS)  for computing the TV distance of two product distributions P and Q where Q is the uniform distribution. This result is extended to the case where Q has a constant number of distinct marginals. In contrast, we show that when P and Q are Bayes net distributions the relative approximation of their TV distance is NP-hard.","doi":"10.24963/ijcai.2023/387","url":"https://ijcai.org/proceedings/2023/387","pdf":"https://ijcai.org/proceedings/2023/0387.pdf","markdown":"https://ijcai.org/proceedings/2023/387.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/387","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":387,"pages":{"first":3479,"last":3487},"keywords":["Machine Learning: ML: Other"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial net- works. In Proc. of ICML, pages 214-223. PMLR, 2017.","title":"Wasserstein generative adversarial networks","authors":"Arjovsky","year":"2017","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bhattacharyya et al., 2020] Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, and N. V. Vinodchandran. Efficient distance approximation for structured high- dimensional distributions via learning. In Proc. of NeurIPS, 2020.","title":"Efficient distance approximation for structured highdimensional distributions via learning","authors":"Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S Meel, N V Vinodchandran","year":"2020","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhattacharyya et al., 2021] Arnab Bhattacharyya, Sutanu Gayen, Saravanan Kandasamy, and N. V. Vinodchandran. Testing product distributions: A closer look. In Algorith- mic Learning Theory, 16-19 March 2021, Virtual Confer- ence, Worldwide, volume 132 of Proceedings of Machine Learning Research, pages 367-396. PMLR, 2021.","title":"Testing product distributions: A closer look","authors":"Arnab Bhattacharyya, Sutanu Gayen, Saravanan Kandasamy, N V Vinodchandran","year":"2021","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bhattacharyya et al., 2022] Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, Dimitrios Myrisiotis, Aduri Pa- van, and N. V. Vinodchandran. On approximating total variation distance. CoRR, abs/2206.07209, 2022.","title":"On approximating total variation distance","authors":"Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S Meel, Dimitrios Myrisiotis, Aduri Pavan, N V Vinodchandran","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cook, 1971] Stephen A. Cook. The complexity of theorem- proving procedures. In Michael A. Harrison, Ranan B. Banerji, and Jeffrey D. Ullman, editors, Proc. of STOC, pages 151-158. ACM, 1971.","title":"The complexity of theoremproving procedures","authors":"Stephen A Cook","year":"1971","venue":"Proc. of STOC","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cortes et al., 2007] Corinna Cortes, Mehryar Mohri, and Ashish Rastogi. L p distance and equivalence of probabilis- tic automata. Int. J. Found. Comput. Sci., 18(4):761-779, 2007.","title":"L p distance and equivalence of probabilistic automata","authors":"Corinna Cortes, Mehryar Mohri, Ashish Rastogi","year":"2007","venue":"Int. J. Found. Comput. Sci","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[de Colnet and Meel, 2019] Alexis de Colnet and Kuldeep S. Meel. Dual hashing-based algorithms for discrete integra- tion. In Proc. of CP, pages 161-176. Springer, 2019.","title":"Alexis de Colnet and Kuldeep S. Meel. Dual hashing-based algorithms for discrete integration","authors":"Colnet De, Meel","year":"2019","venue":"Proc. of CP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dixon et al., 2020] Peter Dixon, Sutanu Gayen, Aduri Pa- van, and N. V. Vinodchandran. Perfect zero knowledge: New upperbounds and relativized separations. In Rafael Pass and Krzysztof Pietrzak, editors, Proc. of TCC, vol- ume 12550 of Lecture Notes in Computer Science, pages 684-704. Springer, 2020.","title":"Perfect zero knowledge: New upperbounds and relativized separations","authors":"Peter Dixon, Sutanu Gayen, Aduri Pavan, N V Vinodchandran","year":"2020","venue":"Proc. of TCC","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dyer et al., 1993] Martin E. Dyer, Alan M. Frieze, Ravi Kannan, Ajai Kapoor, Ljubomir Perkovic, and Umesh V. Vazirani. A mildly exponential time algorithm for ap- proximating the number of solutions to a multidimensional knapsack problem. Comb. Probab. Comput., 2:271-284, 1993.","title":"A mildly exponential time algorithm for approximating the number of solutions to a multidimensional knapsack problem","authors":"Martin E Dyer, Alan M Frieze, Ravi Kannan, Ajai Kapoor, Ljubomir Perkovic, Umesh V Vazirani","year":"1993","venue":"Comb. Probab. Comput","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dyer, 2003] Martin E. Dyer. Approximate counting by dy- namic programming. In Proc. of STOC, June 9-11, 2003, San Diego, CA, USA, pages 693-699. ACM, 2003.","title":"Approximate counting by dynamic programming","authors":"Martin E Dyer","year":"2003","venue":"Proc. of STOC","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Feng et al., 2023] Weiming Feng, Heng Guo, Mark Jerrum, and Jiaheng Wang. A simple polynomial-time approxima- tion algorithm for total variation distances between prod- uct distributions. In Proc. of SOSA, 2023.","title":"A simple polynomial-time approximation algorithm for total variation distances between product distributions","authors":"Weiming Feng, Heng Guo, Mark Jerrum, Jiaheng Wang","year":"2023","venue":"Proc. of SOSA","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Garey and Johnson, 1979] M. R. Garey and David S. John- son. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman, 1979.","title":"Computers and Intractability: A Guide to the Theory of NP-Completeness","authors":"M R Garey, David S Johnson","year":"1979","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Goldreich et al., 1999] Oded Goldreich, Amit Sahai, and Salil P. Vadhan. Can statistical zero knowledge be made non-interactive? or on the relationship of SZK and NISZK. In Proc. of CRYPTO, pages 467-484, 1999.","title":"Can statistical zero knowledge be made non-interactive? or on the relationship of SZK and NISZK","authors":"Oded Goldreich, Amit Sahai, Salil P Vadhan","year":"1999","venue":"Proc. of CRYPTO","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. Advances in neural information processing systems, 27, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gopalan et al., 2010] Parikshit Gopalan, Adam R. Kli- vans, and Raghu Meka. Polynomial-time approximation schemes for knapsack and related counting problems us- ing branching programs. Electron. Colloquium Comput. Complex., page 133, 2010.","title":"Polynomial-time approximation schemes for knapsack and related counting problems using branching programs","authors":"Parikshit Gopalan, Adam R Klivans, Raghu Meka","year":"2010","venue":"Electron. Colloquium Comput. Complex","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kiefer, 2018] Stefan Kiefer. On computing the total varia- tion distance of hidden markov models. In Proc. of ICALP, pages 130:1-130:13, 2018.","title":"On computing the total variation distance of hidden markov models","authors":"Stefan Kiefer, Kiefer","year":"2018","venue":"Proc. of ICALP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lyngsø and Pedersen, 2002] Rune B. Lyngsø and Christian N. S. Pedersen. The consensus string problem and the complexity of comparing hidden markov models. J. Com- put. Syst. Sci., 65(3):545-569, 2002.","title":"The consensus string problem and the complexity of comparing hidden markov models","authors":"B Rune, Lyngsø, N S Christian, Pedersen","year":"2002","venue":"J. Comput. Syst. Sci","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Malka, 2015] Lior Malka. How to achieve perfect simula- tion and a complete problem for non-interactive perfect zero-knowledge. J. Cryptol., 28(3):533-550, 2015.","title":"How to achieve perfect simulation and a complete problem for non-interactive perfect zero-knowledge","authors":"Malka ;, Lior Malka","year":"2015","venue":"J. Cryptol","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Morris and Sinclair, 2004] Ben Morris and Alistair Sinclair. Random walks on truncated cubes and sampling 0-1 Knap- sack solutions. SIAM J. Comput., 34(1):195-226, 2004.","title":"Random walks on truncated cubes and sampling 0-1 Knapsack solutions","authors":"Sinclair ; Morris, Ben Morris, Alistair Sinclair","year":"2004","venue":"SIAM J. Comput","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pearl, 1989] Judea Pearl. Probabilistic reasoning in intel- ligent systems -networks of plausible inference. Morgan Kaufmann series in representation and reasoning. Morgan Kaufmann, 1989.","title":"Judea Pearl. Probabilistic reasoning in intelligent systems -networks of plausible inference","authors":"Pearl","year":"1989","venue":"Morgan Kaufmann series in representation and reasoning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pote and Meel, 2021] Yash Pote and Kuldeep S. Meel. Test- ing probabilistic circuits. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jen- nifer Wortman Vaughan, editors, Advances in Neural In- formation Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 22336-22347, 2021.","title":"Testing probabilistic circuits","authors":"Meel Pote, Yash Pote, Kuldeep S Meel","year":"2021","venue":"Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sahai and Vadhan, 2003] Amit Sahai and Salil P. Vadhan. A complete problem for statistical zero knowledge. J. ACM, 50(2):196-249, 2003.","title":"A complete problem for statistical zero knowledge","authors":"Amit Sahai, P Salil, Vadhan","year":"2003","venue":"J. ACM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Smith et al., 2017] David Smith, Sara Rouhani, and Vibhav Gogate. Order statistics for probabilistic graphical mod- els. In Proceedings of the 26th International Joint Confer- ence on Artificial Intelligence, IJCAI'17, page 4625-4631. AAAI Press, 2017.","title":"Order statistics for probabilistic graphical models","authors":"David Smith, Sara Rouhani, Vibhav Gogate","year":"2017","venue":"Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI'17","doi":null,"url":null,"internal_url":"/proceedings/2017/645"},{"position":24,"raw":"[Stefankovic et al., 2012] Daniel Stefankovic, Santosh S. Vempala, and Eric Vigoda. A deterministic polynomial- time approximation scheme for counting knapsack solu- tions. SIAM J. Comput., 41(2):356-366, 2012.","title":"A deterministic polynomialtime approximation scheme for counting knapsack solutions","authors":"Daniel Stefankovic, S Santosh, Eric Vempala, Vigoda","year":"2012","venue":"SIAM J. Comput","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Stockmeyer, 1976] Larry J. Stockmeyer. The polynomial- time hierarchy. Theor. Comput. Sci., 3(1):1-22, 1976.","title":"The polynomialtime hierarchy","authors":"Larry J Stockmeyer","year":"1976","venue":"Theor. Comput. Sci","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Toda, 1991] Seinosuke Toda. PP is as hard as the polynomial-time hierarchy. SIAM J. Comput., 20(5):865- 877, 1991.","title":"Seinosuke Toda. PP is as hard as the polynomial-time hierarchy","authors":"Toda","year":"1991","venue":"SIAM J. Comput","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}