Proceedings

Bayesian Modelling of Community-Based Multidimensional Trust in Participatory Sensing under Data Sparsity

Download BibTeX Cite

Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 717-723.

Abstract

We propose a new Bayesian model for reliable aggregation of crowdsourced estimates of real-valued quantities in participatory sensing applications. Existing approaches focus on probabilistic modelling of user's reliability as the key to accurate aggregation. However, these are either limited to estimating discrete quantities, or require a significant number of reports from each user to accurately model their reliability. To mitigate these issues, we adopt a community-based approach, which reduces the data required to reliably aggregate real-valued estimates, by leveraging correlations between the reporting behaviour of users belonging to different communities. As a result, our method is up to 16.6% more accurate than existing state-of-the-art methods and is up to 49% more effective under data sparsity when used to estimate Wi-Fi hotspot locations in a real-world crowdsourcing application.

  1. [Bachrach et al., 2012] Y. Bachrach, T. Graepel, T. Minka, and J. Guiver. How to grade a test without knowing the an- swers -a bayesian graphical model for adaptive crowd- sourcing and aptitude testing. In Proceedings of the 29th Int. Conf. on Machine Learning (ICML-12), pages 1183- 1190, July 2012.
  2. [Bishop, 2006] C. Bishop. Pattern recognition and machine learning, volume 4. Springer New York, 2006.
  3. [Dawid and Skene, 1979] A. Dawid and A. Skene. Maxi- mum likelihood estimation of observer error-rates using the em algorithm. Applied statistics, pages 20-28, 1979.
  4. [Despotovic and Aberer, 2005] Z. Despotovic and K. Aberer. Probabilistic prediction of peers' perfor- mance in p2p networks. Engineering Applications of Artificial Intelligence, 18(7):771-780, 2005.
  5. [Gelfand and Smith, 1990] A. Gelfand and A. Smith. Sampling-based approaches to calculating marginal densities. Journal of the American statistical association, 85(410):398-409, 1990.
  6. [Gneiting et al., 2008] T. Gneiting, L. Stanberry, E. Grimit, L. Held, and N. Johnson. Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds. Test, 17(2):211-235, 2008.
  7. [Hall and Jordan, 2010] D. Hall and J. M Jordan. Human- centered information fusion. Artech House, 2010.
  8. [Julier and Uhlmann, 1997] S. J. Julier and J. K. Uhlmann. A non-divergent estimation algorithm in the presence of un- known correlations. In Proceedings of the American Con- trol Conf., pages 2369-2373, 1997.
  9. [Li et al., 2014] Hongwei Li, Bo Zhao, and A. Fuxman. The wisdom of minority: discovering and targeting the right group of workers for crowdsourcing. In Proceedings of the 23rd Int. Conf. on World Wide Web, pages 165-176, 2014.
  10. [Lim et al., 2007] Y. Lim, C.and Wan, B. Ng, and C. See. A real-time indoor wifi localization system utilizing smart antennas. IEEE Transactions on Consumer Electronics, 53(2):618-622, 2007.
  11. [Matheson and Winkler, 1976] J. E Matheson and R. Win- kler. Scoring rules for continuous probability distributions. Management science, 22(10):1087-1096, 1976.
  12. [Minka and Winn, 2009] T. Minka and J. Winn. Gates. In Advances in Neural Information Processing Systems, pages 1073-1080, 2009.
  13. [Minka et al., 2014] T. Minka, J. Winn, J. Guiver, and D. Knowles. Infer.NET 2.6. Microsoft Research Cam- bridge, 2014.
  14. [Naroditskiy et al., 2012] V. Naroditskiy, I. Rahwan, M. Ce- brian, and N. R. Jennings. Verification in referral-based crowdsourcing. PloS One, 7(10):e45924, 2012.
  15. [Oleson et al., 2011] D. Oleson, A. Sorokin, G. Laughlin, V. Hester, J. Le, and L. Biewald. Programmatic gold: Targeted and scalable quality assurance in crowdsourcing. Human computation, 11(11), 2011.
  16. [Pickard et al., 2011] G. Pickard, W. Pan, I. Rahwan, M. Ce- brian, R. Crane, A. Madan, and A. Pentland. Time-critical social mobilization. Science, 334(6055):509-512, 2011.
  17. [Piech et al., 2013] C. Piech, J. Huang, Z. Chen, C. Do, A. Ng, and D. Koller. Tuned models of peer assessment in moocs. In Proceedings of the 6th Int. Conf. on Educa- tional Data Mining (EDM 2013), pages 153-160, 2013.
  18. [Qui et al., 2013] G. Qui, C. Aggarwal, J. Han, and T. Huang. Mining collective intelligence in diverse groups. In 22nd Int. Conf. on World Wide Web (WWW 2013), pages 1041-1052, 2013.
  19. [Raykar et al., 2010] V. Raykar, S. Yu, L. Z., G. Valadez, C. Florin, L. B., and L. Moy. Learning from crowds. The Journal of Machine Learning Research, 11:1297-1322, 2010.
  20. [Salek et al., 2013] M. Salek, Y. Bachrach, and P. Key. Hotspotting-a probabilistic graphical model for image ob- ject localization. In Proceedings of the 27th Conf. on Arti- ficial Intelligence (AAAI), pages 1156-1162, 2013.
  21. [Simpson et al., 2013] E. Simpson, S. Roberts, I. Psorakis, and A. Smith. Dynamic bayesian combination of multiple imperfect classifiers. In Decision Making and Imperfec- tion, pages 1-35. Springer, 2013.
  22. [Teacy et al., 2012] W. T. L. Teacy, M. Luck, A. Rogers, and N. R. Jennings. An efficient and versatile approach to trust and reputation using hierarchical bayesian modelling. Ar- tificial Intelligence, 193:149-185, 2012.
  23. [Tran-Thanh et al., 2013] L. Tran-Thanh, M. Venanzi, A. Rogers, and N. R. Jennings. Efficient budget allocation with accuracy guarantees for crowdsourcing classifica- tion tasks. In Proceedings of the 2013 Int. Conf. on Autonomous Agents and Multi-agent Systems, AAMAS '13, pages 901-908, 2013.
  24. [Venanzi et al., 2013] M. Venanzi, A. Rogers, and N. R. Jen- nings. Trust-based fusion of untrustworthy information in crowdsourcing applications. In Proceedings of the 2013 Int. Conf. on Autonomous Agents and Multi-agent Systems, AAMAS '13, pages 829-836, Richland, SC, 2013.
  25. [Venanzi et al., 2014] M. Venanzi, J. Guiver, G. Kazai, P. Kohli, and M. Shokouhi. Community-based bayesian aggregation models for crowdsourcing. In Proceedings of the 23rd Int. Conf. on World Wide Web, pages 155-164, 2014.
  26. [Wainwright and Jordan, 2008] M. Wainwright and M. Jor- dan. Graphical models, exponential families, and varia- tional inference. Foundations and Trends R in Machine Learning, 1(1-2):1-305, 2008.
  27. [Wang and Wand, 2011] S. Wang and M. P. Wand. Using in- fer. net for statistical analyses. The American Statistician, 65(2), 2011.
  28. [Winn et al., 2005] J. Winn, C. Bishop, and T. Jaakkola. Variational message passing. Journal of Machine Learn- ing Research, 6(4), 2005.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards