{"type":"ScholarlyArticle","title":"GreenFlow: A Computation Allocation Framework for Building Environmentally Sound Recommendation System","authors":[{"name":"Xingyu Lu","slug":"xingyu-lu","given":"Xingyu","family":"Lu","affiliation":"Nanjing University"},{"name":"Zhining Liu","slug":"zhining-liu","given":"Zhining","family":"Liu","affiliation":"Nanjing University"},{"name":"Yanchu Guan","slug":"yanchu-guan","given":"Yanchu","family":"Guan","affiliation":"Nanjing University"},{"name":"Hongxuan Zhang","slug":"hongxuan-zhang","given":"Hongxuan","family":"Zhang","affiliation":"Nanjing University"},{"name":"Chenyi Zhuang","slug":"chenyi-zhuang","given":"Chenyi","family":"Zhuang","affiliation":"Nanjing University"},{"name":"Wenqi Ma","slug":"wenqi-ma","given":"Wenqi","family":"Ma","affiliation":"Nanjing University"},{"name":"Yize Tan","slug":"yize-tan","given":"Yize","family":"Tan","affiliation":"Nanjing University"},{"name":"Jinjie Gu","slug":"jinjie-gu","given":"Jinjie","family":"Gu","affiliation":"Nanjing University"},{"name":"Guannan Zhang","slug":"guannan-zhang","given":"Guannan","family":"Zhang","affiliation":"Nanjing University"}],"abstract":"Given the enormous number of users and items, industrial cascade recommendation systems (RS) are continuously expanded in size and complexity to deliver relevant items, such as news, services, and commodities, to the appropriate users. In a real-world scenario with hundreds of thousands requests per second, significant computation is required to infer personalized results for each request, resulting in a massive energy consumption and carbon emission that raises concern. \r\n\r\nThis paper proposes GreenFlow, a practical computation allocation framework for RS, that considers both accuracy and carbon emission during inference. For each stage (e.g., recall, pre-ranking, ranking, etc.) of a cascade RS, when a user triggers a request, we define two actions that determine the computation: (1) the trained instances of models with different computational complexity; and (2) the number of items to be inferred in the stage. We refer to the combinations of actions in all stages as action chains. A reward score is estimated for each action chain, followed by dynamic primal-dual optimization considering both the reward and computation budget. Extensive experiments verify the effectiveness of the framework, reducing computation consumption by 41% in an industrial mobile application while maintaining commercial revenue. Moreover, the proposed framework saves approximately 5000kWh of electricity and reduces 3 tons of carbon emissions per day.","doi":"10.24963/ijcai.2023/677","url":"https://ijcai.org/proceedings/2023/677","pdf":"https://ijcai.org/proceedings/2023/0677.pdf","markdown":"https://ijcai.org/proceedings/2023/677.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/677","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI for Good","track":"AI for Good","number":677,"pages":{"first":6103,"last":6111},"keywords":["AI for Good: Humans and AI","AI for Good: Machine Learning","AI for Good: Search"],"videos":[],"references":[{"position":1,"raw":"[Aggarwal et al., 2011] Gagan Aggarwal, Gagan Goel, Chinmay Karande, and Aranyak Mehta. Online vertex- weighted bipartite matching and single-bid budgeted allocations. In Proceedings of the twenty-second annual ACM-SIAM symposium on Discrete Algorithms, pages 1253-1264. SIAM, 2011.","title":"Online vertexweighted bipartite matching and single-bid budgeted allocations","authors":"Gagan Aggarwal, Gagan Goel, Chinmay Karande, Aranyak Mehta","year":"2011","venue":"Proceedings of the twenty-second annual ACM-SIAM symposium on Discrete Algorithms","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Agrawal et al., 2014] Shipra Agrawal, Zizhuo Wang, and Yinyu Ye. A dynamic near-optimal algorithm for online linear programming. Operations Research, 62(4):876- 890, 2014.","title":"A dynamic near-optimal algorithm for online linear programming","authors":"Shipra Agrawal, Zizhuo Wang, Yinyu Ye","year":"2014","venue":"Operations Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2012] Kailong Chen, Tianqi Chen, Guoqing Zheng, Ou Jin, Enpeng Yao, and Yong Yu. Collaborative personalized tweet recommendation. In Annual Interna- tional ACM SIGIR Conference on Research and Develop- ment in Information Retrieval, 2012.","title":"Collaborative personalized tweet recommendation","authors":"Kailong Chen, Tianqi Chen, Guoqing Zheng, Ou Jin, Enpeng Yao, Yong Yu","year":"2012","venue":"Annual International ACM SIGIR Conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Covington et al., 2016] Paul Covington, Jay K. Adams, and Emre Sargin. Deep neural networks for youtube recom- mendations. Proceedings of the 10th ACM Conference on Recommender Systems, 2016.","title":"Deep neural networks for youtube recommendations","authors":"Paul Covington, Jay K Adams, Emre Sargin","year":"2016","venue":"Proceedings of the 10th ACM Conference on Recommender Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Han et al., 2016] Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark Horowitz, and William J. Dally. Eie: Efficient inference engine on compressed deep neural network. 2016 ACM/IEEE 43rd Annual In- ternational Symposium on Computer Architecture (ISCA), pages 243-254, 2016.","title":"Efficient inference engine on compressed deep neural network","authors":"Han","year":"2016","venue":"ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Howard et al., 2017] Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mo- bilenets: Efficient convolutional neural networks for mo- bile vision applications. ArXiv, abs/1704.04861, 2017.","title":"Mobilenets: Efficient convolutional neural networks for mobile vision applications","authors":"Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, Hartwig Adam","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2013] Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. Learning deep structured semantic models for web search using clickthrough data. Proceedings of the 22nd ACM interna- tional conference on Information & Knowledge Manage- ment, 2013.","title":"Learning deep structured semantic models for web search using clickthrough data","authors":"Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, Larry Heck","year":"2013","venue":"Proceedings of the 22nd ACM international conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2018] Yanping Huang, Yonglong Cheng, De- hao Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V. Le, and Z. Chen. Gpipe: Efficient training of giant neural net- works using pipeline parallelism. ArXiv, abs/1811.06965, 2018.","title":"Gpipe: Efficient training of giant neural networks using pipeline parallelism","authors":"Yanping Huang, Yonglong Cheng, Dehao Chen, Hyoukjoong Lee, Jiquan Ngiam, Quoc V Le, Z Chen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jiang et al., 2020] Biye Jiang, Pengye Zhang, Rihan Chen, Xinchen Luo, Yin Yang, Guan Wang, Guorui Zhou, Xi- aoqiang Zhu, and Kun Gai. Dcaf: A dynamic computa- tion allocation framework for online serving system. arXiv preprint arXiv:2006.09684, 2020.","title":"Dcaf: A dynamic computation allocation framework for online serving system","authors":"Jiang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Khosravi et al., 2017] Atefeh Khosravi, Lachlan L. H. An- drew, and Rajkumar Buyya. Dynamic vm placement method for minimizing energy and carbon cost in geo- graphically distributed cloud data centers. IEEE Trans- actions on Sustainable Computing, 2:183-196, 2017.","title":"Dynamic vm placement method for minimizing energy and carbon cost in geographically distributed cloud data centers","authors":"Atefeh Khosravi, L H Lachlan, Rajkumar Andrew, Buyya","year":"2017","venue":"IEEE Transactions on Sustainable Computing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kitaev et al., 2020] Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. ArXiv, abs/2001.04451, 2020.","title":"Reformer: The efficient transformer","authors":"Kitaev","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lacoste et al., 2019] Alexandre Lacoste, Alexandra Sasha Luccioni, Victor Schmidt, and Thomas Dandres. Quan- tifying the carbon emissions of machine learning. ArXiv, abs/1910.09700, 2019.","title":"Quantifying the carbon emissions of machine learning","authors":"Alexandre Lacoste, Alexandra Sasha Luccioni, Victor Schmidt, Thomas Dandres","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2020] Xiaocheng Li, Chunlin Sun, and Yinyu Ye. Simple and fast algorithm for binary integer and online lin- ear programming. arXiv preprint arXiv:2003.02513, 2020.","title":"Simple and fast algorithm for binary integer and online linear programming","authors":"Li","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2021] Xin Liu, Mingyu Yan, Lei Deng, Guoqi Li, Xiaochun Ye, and Dongrui Fan. Sampling methods for efficient training of graph convolutional networks: A sur- vey. IEEE/CAA Journal of Automatica Sinica, 9:205-234, 2021.","title":"Sampling methods for efficient training of graph convolutional networks: A survey","authors":"Xin Liu, Mingyu Yan, Lei Deng, Guoqi Li, Xiaochun Ye, Dongrui Fan","year":"2021","venue":"IEEE/CAA Journal of Automatica Sinica","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ma et al., 2018] Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. Entire space multi-task model: An effective approach for estimat- ing post-click conversion rate. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 1137-1140, 2018.","title":"Entire space multi-task model: An effective approach for estimating post-click conversion rate","authors":"Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, Kun Gai","year":"2018","venue":"The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Molchanov et al., 2016] Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. Pruning convo- lutional neural networks for resource efficient inference. arXiv: Learning, 2016.","title":"Pruning convolutional neural networks for resource efficient inference","authors":"Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, Jan Kautz","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pan et al., 2019] Feiyang Pan, Xiang Ao, Pingzhong Tang, Min Lu, Dapeng Liu, Lei Xiao, and Qing He. Field-aware calibration: A simple and empirically strong method for reliable probabilistic predictions. Proceedings of The Web Conference 2020, 2019.","title":"Field-aware calibration: A simple and empirically strong method for reliable probabilistic predictions","authors":"Pan","year":"2019","venue":"Proceedings of The Web Conference 2020","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Qin et al., 2022] Jiarui Qin, Jiachen Zhu, Bo Chen, Zhirong Liu, Weiwen Liu, Ruiming Tang, Rui Zhang, Yong Yu, and Weinan Zhang. Rankflow: Joint optimization of multi- stage cascade ranking systems as flows. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022.","title":"Rankflow: Joint optimization of multistage cascade ranking systems as flows","authors":"Jiarui Qin, Jiachen Zhu, Bo Chen, Zhirong Liu, Weiwen Liu, Ruiming Tang, Rui Zhang, Yong Yu, Weinan Zhang","year":"2022","venue":"Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Schwartz et al., 2019] Roy Schwartz, Jesse Dodge, Noah Smith, and Oren Etzioni. Green ai. Communications of the ACM, 63:54 -63, 2019.","title":"Green ai","authors":"Roy Schwartz, Jesse Dodge, Noah Smith, Oren Etzioni","year":"2019","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shukur et al., 2020] Hanan M. Shukur, Subhi R. M. Zee- baree, Abdulraheem Jamil Ahmed, Rizgar R. Zebari, Omar M. Ahmed, Bareen Shams Aldeen Tahir, and Mo- hammed A. M. Sadeeq. A state of art survey for concur- rent computation and clustering of parallel computing for distributed systems. Journal of Applied Science and Tech- nology Trends, 2020.","title":"A state of art survey for concurrent computation and clustering of parallel computing for distributed systems","authors":"Hanan M Shukur, R M Subhi, Abdulraheem Jamil Zeebaree, Rizgar R Ahmed, Omar M Zebari, Bareen Ahmed, Shams Aldeen, Mohammed A M Tahir, Sadeeq","year":"2020","venue":"Journal of Applied Science and Technology Trends","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tirmazi et al., 2020] Muhammad Tirmazi, Adam Barker, Nan Deng, Md Ehtesam Haque, Zhijing Gene Qin, Steven Hand, Mor Harchol-Balter, and John Wilkes. Borg: the next generation. In EuroSys'20, Heraklion, Crete, 2020.","title":"Borg: the next generation","authors":"Tirmazi","year":"2020","venue":"EuroSys'20","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2011] Lidan Wang, Jimmy J. Lin, and Donald Metzler. A cascade ranking model for efficient ranked re- trieval. Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval, 2011.","title":"A cascade ranking model for efficient ranked retrieval","authors":"Lidan Wang, Jimmy J Lin, Donald Metzler","year":"2011","venue":"Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xu et al., 2021] Jingjing Xu, Wangchunshu Zhou, Zhiyi Fu, Hao Zhou, and Lei Li. A survey on green deep learning. ArXiv, abs/2111.05193, 2021.","title":"A survey on green deep learning","authors":"Jingjing Xu, Wangchunshu Zhou, Zhiyi Fu, Hao Zhou, Lei Li","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2021] Xun Yang, Yunli Wang, Cheng Chen, Qing Tan, Chuan Yu, Jian Xu, and Xiaoqiang Zhu. Com- putation resource allocation solution in recommender sys- tems. ArXiv, abs/2103.02259, 2021.","title":"Computation resource allocation solution in recommender systems","authors":"Xun Yang, Yunli Wang, Cheng Chen, Qing Tan, Chuan Yu, Jian Xu, Xiaoqiang Zhu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yigitcanlar et al., 2021] Tan Yigitcanlar, Rashid Mehmood, and Juan Manuel Corchado. Green artificial intelligence: Towards an efficient, sustainable and equitable technology for smart cities and futures. Sustainability, 2021.","title":"Green artificial intelligence: Towards an efficient, sustainable and equitable technology for smart cities and futures","authors":"Tan Yigitcanlar, Rashid Mehmood, Juan Manuel, Corchado","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2017] Guorui Zhou, Cheng-Ning Song, Xiao- qiang Zhu, Xiao Ma, Yanghui Yan, Xi-Wang Dai, Han Zhu, Junqi Jin, Han Li, and Kun Gai. Deep interest net- work for click-through rate prediction. Proceedings of the 24th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining, 2017.","title":"Deep interest network for click-through rate prediction","authors":"Guorui Zhou, Cheng-Ning Song, Xiaoqiang Zhu, Xiao Ma, Yanghui Yan, Xi-Wang Dai, Han Zhu, Junqi Jin, Han Li, Kun Gai","year":"2017","venue":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2018] Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. Deep interest evolution network for click-through rate pre- diction. In AAAI Conference on Artificial Intelligence, 2018.","title":"Deep interest evolution network for click-through rate prediction","authors":"Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, Kun Gai","year":"2018","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}