{"type":"ScholarlyArticle","title":"Inference with Multinomial Data: Why to Weaken the Prior Strength","authors":[{"name":"Cassio Polpo de Campos","slug":"cassio-polpo-de-campos","given":"Cassio Polpo","family":"de Campos","affiliation":"Dalle Molle Institute for Artificial Intelligence Manno-Lugano , Switzerland"},{"name":"Alessio Benavoli","slug":"alessio-benavoli","given":"Alessio","family":"Benavoli","affiliation":"Dalle Molle Institute for Artificial Intelligence Manno-Lugano , Switzerland"}],"abstract":"This paper considers inference from multinomial data and addresses the problem of choosing the strength of the Dirichlet prior under a mean-squared error criterion. We compare the Maximum Likelihood Estimator (MLE) and the most commonly used Bayesian estimators obtained by assuming a prior Dirichlet distribution with non-informative prior parameters, that is, the parameters of the Dirichlet are equal and altogether sum up to the so called strength of the prior. Under this criterion, MLE becomes more preferable than the Bayesian estimators at the increase of the number of categories k of the multinomial, because non-informative Bayesian estimators induce a region where they are dominant that quickly shrinks with the increase of k. This can be avoided if the strength of the prior is not kept constant but decreased with the number of categories. We argue that the strength should decrease at least k times faster than usual estimators do.","doi":"10.5591/978-1-57735-516-8/IJCAI11-352","url":"https://ijcai.org/proceedings/2011/352","pdf":"https://ijcai.org/proceedings/2011/0352.pdf","markdown":"https://ijcai.org/proceedings/2011/352.md","bibtex":"https://ijcai.org/proceedings/2011/bibtex/352","volume":{"slug":"2011","title":"Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2011","url":"https://ijcai.org/proceedings/2011/","year":2011,"location":"Barcelona, Catalonia, Spain","dates":"16–22 July 2011","editors":"Toby Walsh","isbn":"978-1-57735-512-0; 978-1-57735-513-7; 978-1-57735-514-4; 978-1-57735-515-1; 978-1-57735-516-8","publisher":"AAAI Press/International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":352,"pages":{"first":2107,"last":2111},"keywords":[],"videos":[],"references":[{"position":1,"raw":"[Benavoli and de Campos, 2009] A. Benavoli and C. P. de Campos. Inference from multinomial data based on a MLE-dominance criterion. In Proc. of ECSQARU, pages 22-33. Springer, 2009.","title":"Inference from multinomial data based on a MLE-dominance criterion","authors":"A Benavoli, C P De Campos","year":"2009","venue":"Proc. of ECSQARU","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Berger, 1985] J. O. Berger. Statistical Decision Theory and Bayesian Analysis. Springer Series in Statistics, New York, 1985.","title":"Statistical Decision Theory and Bayesian Analysis","authors":"J O Berger","year":"1985","venue":"Springer Series in Statistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ghosh et al., 1983] M. Ghosh, J.T. Hwang, and K.W Tsui. Construction of improved estimators in multiparameter es- timation for discrete exponential families. Ann. Statist., pages 351-376, 1983.","title":"Construction of improved estimators in multiparameter estimation for discrete exponential families","authors":"M Ghosh, J T Hwang, K W Tsui","year":"1983","venue":"Ann. Statist","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Johnson, 1971] B.M. Johnson. On admissible estimators for certain fixed sample binomial problems. Ann. Math. Statist., 42:1579-1587, 1971.","title":"On admissible estimators for certain fixed sample binomial problems","authors":"B M Johnson","year":"1971","venue":"Ann. Math. Statist","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Koller and Friedman, 2009] D. Koller and N. Friedman. Probabilistic Graphical Models. MIT press, 2009.","title":"Probabilistic Graphical Models","authors":"D Koller, N Friedman","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Mimno and McCallum, 2008] D. Mimno and A. McCal- lum. Topic models conditioned on arbitrary features with Dirichlet-multinomial regression. In Proc. of UAI, pages 411-418. AUAI Press, 2008.","title":"Topic models conditioned on arbitrary features with Dirichlet-multinomial regression","authors":"D Mimno, A Mccallum","year":"2008","venue":"Proc. of UAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Somerville et al., 1997] I. F. Somerville, D. L. Dietrich, and T. A. Mazzuchi. Bayesian reliability analysis using the Dirichlet prior distribution with emphasis on accelerated life testing run in random order. Nonlinear Analysis, 30(7):4415-4423, 1997.","title":"Bayesian reliability analysis using the Dirichlet prior distribution with emphasis on accelerated life testing run in random order","authors":"I F Somerville, D L Dietrich, T A Mazzuchi","year":"1997","venue":"Nonlinear Analysis","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Stein, 1956] C. Stein. Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proc. Third Berkeley Symp. Math. Statist. Prob., pages 197-206.","title":"Inadmissibility of the usual estimator for the mean of a multivariate normal distribution","authors":"C Stein","year":null,"venue":"Proc. Third Berkeley Symp. Math. Statist. Prob","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Zhai and Lafferty, 2001] C. Zhai and J. Lafferty. A study of smoothing methods for language models applied to ad hoc information retrieval. In Proc. ACM SIGIR Conference on Research and development in information retrieval, pages 334-342, New York, NY, USA, 2001. ACM.","title":"A study of smoothing methods for language models applied to ad hoc information retrieval","authors":"C Zhai, J Lafferty","year":"2001","venue":"Proc. ACM SIGIR Conference on Research and development in information retrieval","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}