{"type":"ScholarlyArticle","title":"Parallel Wasserstein Generative Adversarial Nets with Multiple Discriminators","authors":[{"name":"Yuxin Su","slug":"yuxin-su","given":"Yuxin","family":"Su","affiliation":"The Chinese University of Hong Kong , Shatin , Hong Kong"},{"name":"Shenglin Zhao","slug":"shenglin-zhao","given":"Shenglin","family":"Zhao","affiliation":"Youtu Lab , Tencent , Shenzhen , China"},{"name":"Xixian Chen","slug":"xixian-chen","given":"Xixian","family":"Chen","affiliation":"Youtu Lab , Tencent , Shenzhen , China"},{"name":"Irwin King","slug":"irwin-king","given":"Irwin","family":"King","affiliation":"The Chinese University of Hong Kong , Shatin , Hong Kong"},{"name":"Michael Lyu","slug":"michael-lyu","given":"Michael","family":"Lyu","affiliation":"The Chinese University of Hong Kong , Shatin , Hong Kong"}],"abstract":"Wasserstein Generative Adversarial Nets~(GANs) are newly proposed GAN algorithms and widely used in computer vision, web mining, information retrieval, etc. However, the existing algorithms with approximated Wasserstein loss converge slowly due to heavy computation cost and usually generate unstable results as well. In this paper, we solve the computation cost problem by speeding up the Wasserstein GANs from a well-designed communication efficient parallel architecture. Specifically, we develop a new problem formulation targeting the accurate evaluation of Wasserstein distance and propose an easily parallel optimization algorithm to train the Wasserstein GANs. Compared to traditional parallel architecture, our proposed framework is designed explicitly for the skew parameter updates between the generator network and discriminator network. Rigorous experiments reveal that our proposed framework achieves a significant improvement regarding convergence speed with comparable stability on generating images, compared to the state-of-the-art of Wasserstein GANs algorithms.","doi":"10.24963/ijcai.2019/483","url":"https://ijcai.org/proceedings/2019/483","pdf":"https://ijcai.org/proceedings/2019/0483.pdf","markdown":"https://ijcai.org/proceedings/2019/483.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/483","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":483,"pages":{"first":3483,"last":3489},"keywords":["Machine Learning: Adversarial Machine Learning","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martín Abadi, Paul Barham, et al. Ten- sorflow: A system for large-scale machine learning. In OSDI, pages 265-283, 2016.","title":"Tensorflow: A system for large-scale machine learning","authors":"Martín Abadi, Paul Barham","year":"2016","venue":"OSDI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Agueh and Carlier, 2011] Martial Agueh and Guillaume Carlier. Barycenters in the wasserstein space. SIAM J. Math. Analysis, 43(2):904-924, 2011.","title":"Martial Agueh and Guillaume Carlier. Barycenters in the wasserstein space","authors":"Carlier Agueh","year":"2011","venue":"SIAM J. Math. Analysis","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Arjovsky et al., 2017] Martín Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial net- works. In ICML, pages 214-223, 2017.","title":"Martín Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks","authors":"Arjovsky","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bousquet et al., 2017] Olivier Bousquet, Sylvain Gelly, Ilya Tolstikhin, Carl Johann Simon-Gabriel, and Bernhard Schölkopf. From optimal transport to generative model- ing: the vegan cookbook. Technical report, 2017.","title":"From optimal transport to generative modeling: the vegan cookbook","authors":"Olivier Bousquet, Sylvain Gelly, Ilya Tolstikhin, Carl Johann Simon-Gabriel, Bernhard Schölkopf","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chavdarova and Fleuret, 2018] Tatjana Chavdarova and Franc ¸ois Fleuret. Sgan: An alternative training of generative adversarial networks. In CVPR, 2018.","title":"Sgan: An alternative training of generative adversarial networks","authors":"Fleuret Chavdarova, Tatjana Chavdarova, Franc ¸ois, Fleuret","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Claici et al., 2018] Sebastian Claici, Edward Chien, and Justin Solomon. Stochastic wasserstein barycenters. In ICML, pages 998-1007, 2018.","title":"Stochastic wasserstein barycenters","authors":"Sebastian Claici, Edward Chien, Justin Solomon","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cuturi, 2013] Marco Cuturi. Sinkhorn distances: Light- speed computation of optimal transport. In NeurIPS, pages 2292-2300, 2013.","title":"Sinkhorn distances: Lightspeed computation of optimal transport","authors":"Marco Cuturi","year":"2013","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dean and et al., 2012] Jeffrey Dean and Greg Corrado et al. Large scale distributed deep networks. In NeurIPS, pages 1232-1240, 2012.","title":"Large scale distributed deep networks","authors":"Jeffrey Dean, Greg Corrado","year":"2012","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Deshpande et al., 2018] Ishan Deshpande, Ziyu Zhang, and Alexander G. Schwing. Generative modeling using the sliced wasserstein distance. In CVPR, June 2018.","title":"Generative modeling using the sliced wasserstein distance","authors":"Ishan Deshpande, Ziyu Zhang, Alexander G Schwing","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Genevay et al., 2016] Aude Genevay, Marco Cuturi, Gabriel Peyré, and Francis R. Bach. Stochastic optimiza- tion for large-scale optimal transport. In NeurIPS, pages 3432-3440, 2016.","title":"Stochastic optimization for large-scale optimal transport","authors":"Aude Genevay, Marco Cuturi, Gabriel Peyré, Francis R Bach","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Genevay et al., 2018a] Aude Genevay, Lénaic Chizat, Fran- cis Bach, Marco Cuturi, and Gabriel Peyré. Sam- ple complexity of sinkhorn divergences. arXiv preprint arXiv:1810.02733, 2018.","title":null,"authors":"Genevay","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Genevay et al., 2018b] Aude Genevay, Gabriel Peyré, and Marco Cuturi. Learning generative models with sinkhorn divergences. In AISTATS, pages 1608-1617, 2018.","title":"Learning generative models with sinkhorn divergences","authors":"Genevay","year":"2018","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS, pages 2672- 2680, 2014.","title":"Generative adversarial nets","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C Courville, Yoshua Bengio","year":"2014","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gulrajani et al., 2017] Ishaan Gulrajani, Faruk Ahmed, Martín Arjovsky, Vincent Dumoulin, and Aaron C. Courville. Improved training of wasserstein gans. In NeurIPS, pages 5769-5779, 2017.","title":"Improved training of wasserstein gans","authors":"Ishaan Gulrajani, Faruk Ahmed, Martín Arjovsky, Aaron C Vincent Dumoulin, Courville","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Heusel et al., 2017] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochre- iter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS, pages 6629-6640, 2017.","title":"Gans trained by a two time-scale update rule converge to a local nash equilibrium","authors":"Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Sepp Hochreiter","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2016] Jinfeng Li, James Cheng, Yunjian Zhao, Fan Yang, Yuzhen Huang, Haipeng Chen, and Ruihao Zhao. A comparison of general-purpose distributed systems for data processing. In Big Data, pages 378-383, 2016.","title":"A comparison of general-purpose distributed systems for data processing","authors":"Li","year":"2016","venue":"Big Data","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Paszke et al., 2017] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De- Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.","title":"Automatic differentiation in pytorch","authors":"Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De-Vito, Zeming Lin, Alban Desmaison, Luca Antiga, Adam Lerer","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Petzka et al., 2018] Henning Petzka, Asja Fischer, and De- nis Lukovnicov. On the regularization of wasserstein gans. ICLR, 2018.","title":"On the regularization of wasserstein gans","authors":"Asja Henning Petzka, Denis Fischer, Lukovnicov","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Salimans et al., 2018] Tim Salimans, Han Zhang, Alec Radford, and Dimitris N. Metaxas. Improving gans using optimal transport. ICLR, 2018.","title":"Improving gans using optimal transport","authors":"Tim Salimans, Han Zhang, Alec Radford, Dimitris N Metaxas","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Santambrogio, 2015] Filippo Santambrogio. Optimal trans- port for applied mathematicians. Birkäuser, NY, 55:58-63, 2015.","title":"Optimal transport for applied mathematicians","authors":"Filippo Santambrogio","year":"2015","venue":"Birkäuser","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Staib et al., 2017] Matthew Staib, Sebastian Claici, Justin M. Solomon, and Stefanie Jegelka. Parallel streaming wasserstein barycenters. In NeurIPS, pages 2644-2655, 2017.","title":"Parallel streaming wasserstein barycenters","authors":"Matthew Staib, Sebastian Claici, Justin M Solomon, Stefanie Jegelka","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Su et al., 2017] Yuxin Su, Irwin King, and Michael R. Lyu. Learning to rank using localized geometric mean metrics. In SIGIR, pages 45-54, 2017.","title":"Learning to rank using localized geometric mean metrics","authors":"Yuxin Su, Irwin King, Michael R Lyu","year":"2017","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Villani, 2008] Cédric Villani. Optimal transport: old and new, volume 338. Springer Science & Business Media, 2008.","title":"Optimal transport: old and new","authors":"Villani","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017] Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang. IRGAN: A minimax game for unifying gen- erative and discriminative information retrieval models. In SIGIR, pages 515-524, 2017.","title":"IRGAN: A minimax game for unifying generative and discriminative information retrieval models","authors":"Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, Dell Zhang","year":"2017","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wei et al., 2018] Xiang Wei, Boqing Gong, Zixia Liu, Wei Lu, and Liqiang Wang. Improving the improved training of wasserstein gans: A consistency term and its dual effect. ICLR, 2018.","title":"Improving the improved training of wasserstein gans: A consistency term and its dual effect","authors":"Xiang Wei, Boqing Gong, Zixia Liu, Wei Lu, Liqiang Wang","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2018] Jiqing Wu, Zhiwu Huang, Janine Thoma, Dinesh Acharya, and Luc Van Gool. Wasserstein diver- gence for gans. ECCV, 2018.","title":"Wasserstein divergence for gans","authors":"Jiqing Wu, Zhiwu Huang, Janine Thoma, Dinesh Acharya, Luc Van Gool","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2015] Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. LSUN: construction of a large- scale image dataset using deep learning with humans in the loop. CoRR, abs/1506.03365, 2015.","title":"LSUN: construction of a largescale image dataset using deep learning with humans in the loop","authors":"Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, Jianxiong Xiao","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2019] Jiani Zhang, Xingjian Shi, Shenglin Zhao, and Irwin King. Star-gcn: Stacked and recon- structed graph convolutional networks for recommender systems. arXiv preprint arXiv:1905.13129, 2019.","title":"Star-gcn: Stacked and reconstructed graph convolutional networks for recommender systems","authors":"Jiani Zhang, Xingjian Shi, Shenglin Zhao, Irwin King","year":"2019","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2019/592"},{"position":29,"raw":"[Zhao et al., 2016] Shenglin Zhao, Tong Zhao, Haiqin Yang, Michael R Lyu, and Irwin King. Stellar: spatial-temporal latent ranking for successive point-of-interest recommen- dation. In AAAI, 2016.","title":"Stellar: spatial-temporal latent ranking for successive point-of-interest recommendation","authors":"Shenglin Zhao, Tong Zhao, Haiqin Yang, Irwin Michael R Lyu, King","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhao et al., 2018] Shenglin Zhao, Xixian Chen, Irwin King, and Michael R Lyu. Personalized sequential check-in pre- diction: Beyond geographical and temporal contexts. In ICME, pages 1-6, 2018.","title":"Personalized sequential check-in prediction: Beyond geographical and temporal contexts","authors":"Zhao","year":"2018","venue":"ICME","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}