{"type":"ScholarlyArticle","title":"Data Valuation in Machine Learning: \"Ingredients\", Strategies, and Open Challenges","authors":[{"name":"Rachael Hwee Ling Sim","slug":"rachael-hwee-ling-sim","given":"Rachael Hwee Ling","family":"Sim","affiliation":"Department of Computer Science , National University of Singapore"},{"name":"Xinyi Xu","slug":"xinyi-xu","given":"Xinyi","family":"Xu","affiliation":"Department of Computer Science , National University of Singapore"},{"name":"Bryan Kian Hsiang Low","slug":"bryan-kian-hsiang-low","given":"Bryan Kian Hsiang","family":"Low","affiliation":"Department of Computer Science , National University of Singapore"}],"abstract":"Data valuation in machine learning (ML) is an emerging research area that studies the worth of data in ML. Data valuation is used in collaborative ML to determine a fair compensation for every data owner and in interpretable ML to identify the most responsible, noisy, or misleading training examples. This paper presents a comprehensive technical survey that provides a new formal study of data valuation in ML through its “ingredients” and the corresponding properties, grounds the discussion of common desiderata satisfied by existing data valuation strategies on our proposed ingredients, and identifies open research challenges for designing new ingredients, data valuation strategies, and cost reduction techniques.","doi":"10.24963/ijcai.2022/782","url":"https://ijcai.org/proceedings/2022/782","pdf":"https://ijcai.org/proceedings/2022/0782.pdf","markdown":"https://ijcai.org/proceedings/2022/782.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/782","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Survey Track","track":"Survey Track","number":782,"pages":{"first":5607,"last":5614},"keywords":["Survey Track: Multidisciplinary Topics and Applications","Survey Track: AI Ethics, Trust, Fairness","Survey Track: Machine Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987306","length_sec":63,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985449","length_sec":685,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martin Abadi, Andy Chu, Ian Goodfel- low, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proc. ACM CCS, 2016.","title":"Deep learning with differential privacy","authors":"Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan Mcmahan, Ilya Mironov, Kunal Talwar, Li Zhang","year":"2016","venue":"Proc. ACM CCS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Abay et al., 2019] Nazmiye Ceren Abay, Yan Zhou, Mu- rat Kantarcioglu, Bhavani Thuraisingham, and Latanya Sweeney. Privacy preserving synthetic data release using deep learning. In Machine Learning and Knowledge Dis- covery in Databases, pages 510-526, 2019.","title":"Privacy preserving synthetic data release using deep learning","authors":"Abay","year":"2019","venue":"Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Agarwal et al., 2019] Anish Agarwal, Munther Dahleh, and Tuhin Sarkar. A marketplace for data: An algorithmic so- lution. In Proc. EC, pages 701-726, 2019.","title":"A marketplace for data: An algorithmic solution","authors":"Anish Agarwal, Munther Dahleh, Tuhin Sarkar","year":"2019","venue":"Proc. EC","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Basu et al., 2021] Samyadeep Basu, Philip Pope, and Soheil Feizi. Influence functions in deep learning are fragile. In Proc. ICLR, 2021.","title":"Influence functions in deep learning are fragile","authors":"Basu","year":"2021","venue":"Proc. ICLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bax, 2019] Eric Bax. Computing a data dividend. arXiv:1905.01805, 2019.","title":"Computing a data dividend","authors":"Eric Bax, Bax","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bian et al., 2021] Yatao Bian, Yu Rong, Tingyang Xu, Jiaxiang Wu, Andreas Krause, and Junzhou Huang. Energy-based learning for cooperative games, with ap- plications to valuation problems in machine learning. arXiv:2106.02938, 2021.","title":"Energy-based learning for cooperative games, with applications to valuation problems in machine learning","authors":"Yatao Bian, Yu Rong, Tingyang Xu, Jiaxiang Wu, Andreas Krause, Junzhou Huang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chalkiadakis et al., 2011] Georgios Chalkiadakis, Edith Elkind, and Michael Wooldridge. Computational aspects of cooperative game theory. In R. J. Brachman, W. W. Co- hen, and T. G. Dietterich, editors, Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, 2011.","title":"Computational aspects of cooperative game theory","authors":"Georgios Chalkiadakis, Edith Elkind, Michael Wooldridge","year":"2011","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cong et al., 2021] Zicun Cong, Xuan Luo, Pei Jian, Feida Zhu, and Yong Zhang. Data pricing in machine learning pipelines. arXiv:2108.07915, 2021.","title":"Data pricing in machine learning pipelines","authors":"Cong","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cook and Weisberg, 1980] R. Dennis Cook and Sanford Weisberg. Characterizations of an empirical influence function for detecting influential cases in regression. Tech- nometrics, 22(4):495-508, 1980.","title":"Characterizations of an empirical influence function for detecting influential cases in regression","authors":"R, Dennis Cook, Sanford Weisberg","year":"1980","venue":"Technometrics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Devereaux et al., 2016] P.J. Devereaux, Gordon Guyatt, Hertzel Gerstein, Stuart Connolly, and Salim Yusuf. To- ward fairness in data sharing. New England Journal of Medicine, 375(5):405-407, 2016.","title":"Toward fairness in data sharing","authors":"P J Devereaux, Gordon Guyatt, Hertzel Gerstein, Stuart Connolly, Salim Yusuf","year":"2016","venue":"New England Journal of Medicine","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fatima et al., 2008] Shaheen S Fatima, Michael Wooldridge, and Nicholas R Jennings. A linear ap- proximation method for the Shapley value. Artificial Intelligence, 172(14):1673-1699, 2008.","title":"A linear approximation method for the Shapley value","authors":"Michael Shaheen S Fatima, Nicholas R Wooldridge, Jennings","year":"2008","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fricker and Maksimov, 2017] Samuel A Fricker and Yuliyan V Maksimov. Pricing of data products in data marketplaces. In Proc. ICSOB, pages 49-66, 2017.","title":"Pricing of data products in data marketplaces","authors":"Maksimov ; Fricker, A Samuel, Yuliyan V Fricker, Maksimov","year":"2017","venue":"Proc. ICSOB","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ghorbani and Zou, 2019] Amirata Ghorbani and James Zou. Data Shapley: Equitable valuation of data for ma- chine learning. In Proc. ICML, pages 2242-2251, 2019.","title":"Data Shapley: Equitable valuation of data for machine learning","authors":"Zou Ghorbani, Amirata Ghorbani, James Zou","year":"2019","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ghorbani et al., 2020] Amirata Ghorbani, Michael Kim, and James Zou. A distributional framework for data valu- ation. In Proc. ICML, 2020.","title":"A distributional framework for data valuation","authors":"Ghorbani","year":"2020","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger, editors, Proc. NeurIPS, volume 27, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Heckman et al., 2015] Judd Randolph Heckman, Erin Lau- rel Boehmer, Elizabeth Hope Peters, Milad Davaloo, and Nikhil Gopinath Kurup. A pricing model for data markets. In Proc. iConference, 2015.","title":"A pricing model for data markets","authors":"Judd Randolph Heckman, Erin Laurel Boehmer, Elizabeth Hope Peters, Milad Davaloo, Nikhil Gopinath, Kurup","year":"2015","venue":"Proc. iConference","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jia et al., 2019a] Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nezihe Merve Gurel, Bo Li, Ce Zhang, Costas J Spanos, and Dawn Song. Effi- cient task-specific data valuation for nearest neighbor al- gorithms. Proc. VLDB Endowment, 12(11):1610-1623, 2019.","title":"Efficient task-specific data valuation for nearest neighbor algorithms","authors":"Jia","year":"2019","venue":"Proc. VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Jia et al., 2019b] Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas J Spanos. To- wards efficient data valuation based on the Shapley value. In Proc. AISTATS, pages 1167-1176, 2019.","title":"Towards efficient data valuation based on the Shapley value","authors":"Jia","year":"2019","venue":"Proc. AISTATS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jia et al., 2019c] Ruoxi Jia, Xuehui Sun, Jiacen Xu, Ce Zhang, Bo Li, and Dawn Song. An empirical and comparative analysis of data valuation with scalable algo- rithms. arXiv:1911.07128, 2019.","title":"An empirical and comparative analysis of data valuation with scalable algorithms","authors":"Jia","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kingma and Welling, 2014] Diederik P Kingma and Max Welling. Auto-encoding variational Bayes. In Proc. ICLR, 2014.","title":"Auto-encoding variational Bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"Proc. ICLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Koenecke and Varian, 2020] Allison Koenecke and Hal Varian. Synthetic data generation for economists. arXiv:2011.01374, 2020.","title":"Synthetic data generation for economists","authors":"Varian Koenecke, Allison Koenecke, Hal Varian","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Koh and Liang, 2017] Pang Wei Koh and Percy Liang. Un- derstanding black-box predictions via influence functions. In Proc. ICML, pages 1885-1894, 2017.","title":"Understanding black-box predictions via influence functions","authors":"Pang Wei, Koh, Percy Liang","year":"2017","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Koh et al., 2019] Pang Wei Koh, Kai-Siang Ang, Hubert Teo, and Percy S Liang. On the accuracy of influence functions for measuring group effects. In Proc. NeurIPS, volume 32, 2019.","title":"On the accuracy of influence functions for measuring group effects","authors":"Pang Wei Koh, Kai-Siang Ang, Hubert Teo, Percy S Liang","year":"2019","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Krause and Guestrin, 2007] Andreas Krause and Carlos Guestrin. Nonmyopic active learning of Gaussian pro- cesses: an exploration-exploitation approach. In Proc. ICML, pages 449-456, 2007.","title":"Nonmyopic active learning of Gaussian processes: an exploration-exploitation approach","authors":"Guestrin Krause, Andreas Krause, Carlos Guestrin","year":"2007","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Kruschke, 2008] John Kruschke. Bayesian approaches to associative learning: From passive to active learning. Learning & Behavior, 36(3):210-226, 2008.","title":"John Kruschke. Bayesian approaches to associative learning: From passive to active learning","authors":"Kruschke","year":"2008","venue":"Learning & Behavior","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Kwon and Zou, 2021] Yongchan Kwon and James Zou. Beta Shapley: a unified and noise-reduced data valuation framework for machine learning. In Proc. AISTATS, 2021.","title":"Beta Shapley: a unified and noise-reduced data valuation framework for machine learning","authors":"Zou Kwon, Yongchan Kwon, James Zou","year":"2021","venue":"Proc. AISTATS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Liang et al., 2018] Fan Liang, Wei Yu, Dou An, Qingyu Yang, Xinwen Fu, and Wei Zhao. A survey on big data market: Pricing, trading and protection. IEEE Access, 6:15132-15154, 2018.","title":"A survey on big data market: Pricing, trading and protection","authors":"Fan Liang, Wei Yu, Dou An, Qingyu Yang, Xinwen Fu, Wei Zhao","year":"2018","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Maleki et al., 2013] Sasan Maleki, Long Tran-Thanh, Greg Hines, Talal Rahwan, and Alex Rogers. Bounding the esti- mation error of sampling-based Shapley value approxima- tion. arXiv:1306.4265, 2013.","title":"Bounding the estimation error of sampling-based Shapley value approximation","authors":"Sasan Maleki, Long Tran-Thanh, Greg Hines, Talal Rahwan, Alex Rogers","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Maschler and Peleg, 1966] Michael Maschler and Bezalel Peleg. A characterization, existence proof and dimension bounds for the kernel of a game. Pacific J. Mathematics, 18(2):289-328, 1966.","title":"A characterization, existence proof and dimension bounds for the kernel of a game","authors":"Peleg ; Maschler, Bezalel Peleg","year":"1966","venue":"Pacific J. Mathematics","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Mitchell et al., 2022] Rory Mitchell, Joshua Cooper, Eibe Frank, and Geoffrey Holmes. Sampling permutations for shapley value estimation. Journal of Machine Learning Research, 23(43):1-46, 2022.","title":"Sampling permutations for shapley value estimation","authors":"Rory Mitchell, Joshua Cooper, Eibe Frank, Geoffrey Holmes","year":"2022","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Muschalle et al., 2013] Alexander Muschalle, Florian Stahl, Alexander Löser, and Gottfried Vossen. Pricing approaches for data markets. In Malu Castellanos, Umesh- war Dayal, and Elke A. Rundensteiner, editors, Enabling Real-Time Business Intelligence, pages 129-144, Berlin, Heidelberg, 2013. Springer Berlin Heidelberg.","title":"Pricing approaches for data markets","authors":"Alexander Muschalle, Florian Stahl, Alexander Löser, Gottfried Vossen","year":"2013","venue":"Enabling Real-Time Business Intelligence","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Ohrimenko et al., 2019] Olga Ohrimenko, Shruti Tople, and Sebastian Tschiatschek. Collaborative machine learning markets with data-replication-robust payments. In Proc. NeurIPS SGO & ML Workshop, 2019.","title":"Collaborative machine learning markets with data-replication-robust payments","authors":"Olga Ohrimenko, Shruti Tople, Sebastian Tschiatschek","year":"2019","venue":"Proc. NeurIPS SGO & ML Workshop","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Pei, 2020] Jian Pei. A survey on data pricing: from eco- nomics to data science. IEEE Transactions on Knowledge & Data Engineering, 2020.","title":"A survey on data pricing: from economics to data science","authors":"Jian Pei","year":"2020","venue":"IEEE Transactions on Knowledge & Data Engineering","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Raskar et al., 2019] Ramesh Raskar, Praneeth Vepakomma, Tristan Swedish, and Aalekh Sharan. Data markets to support AI for all: Pricing, valuation and governance. arXiv:1905.06462, 2019.","title":"Data markets to support AI for all: Pricing, valuation and governance","authors":"Ramesh Raskar, Praneeth Vepakomma, Tristan Swedish, Aalekh Sharan","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Richardson et al., 2020] Adam Richardson, Aris Filos- Ratsikas, and Boi Faltings. Budget-bounded incentives for Federated learning. In Federated Learning, pages 176- 188. Springer, 2020.","title":"Budget-bounded incentives for Federated learning","authors":"Adam Richardson, Aris Filos-Ratsikas, Boi Faltings","year":"2020","venue":"Federated Learning","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Rozemberczki and Sarkar, 2021] Benedek Rozemberczki and Rik Sarkar. The Shapley value of classifiers in ensemble games. In ACM Symposium on Neural Gaze Detection, 2021.","title":"Benedek Rozemberczki and Rik Sarkar. The Shapley value of classifiers in ensemble games","authors":"Sarkar Rozemberczki","year":"2021","venue":"ACM Symposium on Neural Gaze Detection","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Shapley, 1953] Lloyd Shapley. A value for n-person games.","title":"Lloyd Shapley. A value for n-person games","authors":"Shapley","year":"1953","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"In H. W. Kuhn and A. W. Tucker, editors, Contributions to the Theory of Games, volume 2, pages 307-317. Princeton Univ. Press, 1953.","title":"Contributions to the Theory of Games","authors":null,"year":"1953","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Sim et al., 2020] Rachael Hwee Ling Sim, Yehong Zhang, Mun Choon Chan, and Bryan Kian Hsiang Low. Collab- orative machine learning with incentive-aware model re- wards. In Proc. ICML, pages 8927-8936, 2020.","title":"Collaborative machine learning with incentive-aware model rewards","authors":"Rachael Hwee, Ling Sim, Yehong Zhang, Mun Choon Chan, Bryan Kian, Hsiang Low","year":"2020","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Singhal et al., 2021] Karan Singhal, Hakim Sidahmed, Zachary Garrett, Shanshan Wu, John Rush, and Sushant Prakash. Federated reconstruction: Partially local feder- ated learning. In Proc. NeurIPS, 2021.","title":"Federated reconstruction: Partially local federated learning","authors":"Karan Singhal, Hakim Sidahmed, Zachary Garrett, Shanshan Wu, John Rush, Sushant Prakash","year":"2021","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Song et al., 2019] Tianshu Song, Yongxin Tong, and Shuyue Wei. Profit allocation for Federated learning. In Proc. IEEE Big Data, pages 2577-2586. IEEE, 2019.","title":"Profit allocation for Federated learning","authors":"Tianshu Song, Yongxin Tong, Shuyue Wei","year":"2019","venue":"Proc. IEEE Big Data","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Stahl and Vossen, 2016] Florian Stahl and Gottfried Vossen. Data quality scores for pricing on data marketplaces. In N.T. Nguyen, G. Trawiński, H. Fujita, and T-P. Hong, edi- tors, Intelligent Information and Database Systems, pages 215-224, Berlin, Heidelberg, 2016. Springer Berlin Hei- delberg.","title":"Data quality scores for pricing on data marketplaces","authors":"Vossen Stahl, Florian Stahl, Gottfried Vossen","year":"2016","venue":"Intelligent Information and Database Systems","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Tay et al., 2022] Sebastian Shenghong Tay, Xinyi Xu, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Incen- tivizing collaboration in machine learning via synthetic data rewards. In Proc. AAAI, 2022.","title":"Incentivizing collaboration in machine learning via synthetic data rewards","authors":"Sebastian Shenghong Tay, Xinyi Xu, Chuan Sheng Foo, Bryan Kian, Hsiang Low","year":"2022","venue":"Proc. AAAI","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wang et al., 2021] Tianhao Wang, Yu Yang, and Ruoxi Jia. Learnability of learning performance and its application to data valuation. arXiv:2107.06336, 2021.","title":"Learnability of learning performance and its application to data valuation","authors":"Tianhao Wang, Yu Yang, Ruoxi Jia","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Xu et al., 2021a] Xinyi Xu, Lingjuan Lyu, Xingjun Ma, Chenglin Miao, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Gradient driven rewards to guarantee fairness in col- laborative machine learning. In Proc. NeurIPS, volume 34, 2021.","title":"Gradient driven rewards to guarantee fairness in collaborative machine learning","authors":"Xu","year":"2021","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Xu et al., 2021b] Xinyi Xu, Zhaoxuan Wu, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Validation free and replication robust volume-based data valuation. In Proc. NeurIPS, volume 34, 2021.","title":"Validation free and replication robust volume-based data valuation","authors":"Xu","year":"2021","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Yan and Procaccia, 2021] Tom Yan and Ariel D Procaccia. If you like Shapley then you'll love the core. In Proc. AAAI, pages 5751-5759, 2021.","title":"If you like Shapley then you'll love the core","authors":"Yan, Procaccia ; ] Tom Yan, Ariel D Procaccia","year":"2021","venue":"Proc. AAAI","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Yoon et al., 2020] Jinsung Yoon, Sercan Arik, and Tomas Pfister. Data valuation using reinforcement learning. In Proc. ICML, 2020.","title":"Data valuation using reinforcement learning","authors":"Yoon","year":"2020","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Zhang and Beltrán, 2020] Mengxiao Zhang and Fer- nando Beltrán. A survey of data pricing methods. SSRN:3609120, 2020.","title":"Mengxiao Zhang and Fernando Beltrán. A survey of data pricing methods","authors":"Beltrán Zhang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Zhao et al., 2020] Yi Zhao, Ke Xu, Feng Yan, Yuchao Zhang, Yao Fu, and Haiyang Wang. Auction-based High Timeliness Data Pricing under Mobile and Wireless Net- works. IEEE International Conference on Communica- tions, 2020.","title":"Auction-based High Timeliness Data Pricing under Mobile and Wireless Networks","authors":"Yi Zhao, Ke Xu, Feng Yan, Yuchao Zhang, Yao Fu, Haiyang Wang","year":"2020","venue":"IEEE International Conference on Communications","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}