{"type":"ScholarlyArticle","title":"LocMoE: A Low-overhead MoE for Large Language Model Training","authors":[{"name":"Jing Li","slug":"jing-li","given":"Jing","family":"Li","affiliation":"Huawei Technologies Co., Ltd {lijing473 ,"},{"name":"Zhijie Sun","slug":"zhijie-sun","given":"Zhijie","family":"Sun","affiliation":"Huawei Technologies Co., Ltd {lijing473 ,"},{"name":"Xuan He","slug":"xuan-he","given":"Xuan","family":"He","affiliation":"Huawei Technologies Co., Ltd {lijing473 ,"},{"name":"Li Zeng","slug":"li-zeng","given":"Li","family":"Zeng","affiliation":"Huawei Technologies Co., Ltd {lijing473 ,"},{"name":"Yi Lin","slug":"yi-lin","given":"Yi","family":"Lin","affiliation":"Huawei Technologies Co., Ltd {lijing473 ,"},{"name":"Entong Li","slug":"entong-li","given":"Entong","family":"Li","affiliation":"Huawei Technologies Co., Ltd {lijing473 ,"},{"name":"Binfan Zheng","slug":"binfan-zheng","given":"Binfan","family":"Zheng","affiliation":"Huawei Technologies Co., Ltd {lijing473 ,"},{"name":"Rongqian Zhao","slug":"rongqian-zhao","given":"Rongqian","family":"Zhao","affiliation":"Huawei Technologies Co., Ltd {lijing473 ,"},{"name":"Xin Chen","slug":"xin-chen","given":"Xin","family":"Chen","affiliation":"Huawei Technologies Co., Ltd {lijing473 ,"}],"abstract":"The Mixtures-of-Experts (MoE) model is a widespread distributed and integrated learning method for large language models (LLM), which is favored due to its ability to sparsify and expand models efficiently. However, the performance of MoE is limited by load imbalance and high latency of All-to-All communication, along with relatively redundant computation owing to large expert capacity. Load imbalance may result from existing routing policies that consistently tend to select certain experts. The frequent inter-node communication in the All-to-All procedure also significantly prolongs the training time. To alleviate the above performance problems, we propose a novel routing strategy that combines load balance and locality by converting partial inter-node communication to that of intra-node. Notably, we elucidate that there is a minimum threshold for expert capacity, calculated through the maximal angular deviation between the gating weights of the experts and the assigned tokens. We port these modifications on the PanGu-Σ model based on the MindSpore framework with multi-level routing and conduct experiments on Ascend clusters. The experiment results demonstrate that the proposed LocMoE reduces training time per epoch by 12.68% to 22.24% compared to classical routers, such as hash router and switch router, without impacting the model accuracy.","doi":"10.24963/ijcai.2024/705","url":"https://ijcai.org/proceedings/2024/705","pdf":"https://ijcai.org/proceedings/2024/0705.pdf","markdown":"https://ijcai.org/proceedings/2024/705.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/705","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Natural Language Processing","track":"Main Track","number":705,"pages":{"first":6377,"last":6387},"keywords":["Natural Language Processing: NLP: Other","Natural Language Processing: NLP: Interpretability and analysis of models for NLP","Natural Language Processing: NLP: Language models"],"videos":[],"references":[{"position":1,"raw":"[Bi et al., 2023] Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium- range global weather forecasting with 3d neural networks. Nature, 619(7970):533-538, 2023.","title":"Accurate mediumrange global weather forecasting with 3d neural networks","authors":"Bi","year":"2023","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Brown et al., 2020] Tom Brown, Benjamin Mann, Nick Ry- der, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing sys- tems, 33:1877-1901, 2020.","title":"Language models are few-shot learners","authors":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chi et al., 2022] Zewen Chi, Li Dong, Shaohan Huang, Damai Dai, Shuming Ma, Barun Patra, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, et al. On the repre- sentation collapse of sparse mixture of experts. Advances in Neural Information Processing Systems, 35:34600- 34613, 2022.","title":"On the representation collapse of sparse mixture of experts","authors":"Chi","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chowdhury et al., 2023] Mohammed Nowaz Rabbani Chowdhury, Shuai Zhang, Meng Wang, Sijia Liu, and Pin-Yu Chen. Patch-level routing in mixture-of-experts is provably sample-efficient for convolutional neural networks. arXiv preprint arXiv:2306.04073, 2023.","title":"Patch-level routing in mixture-of-experts is provably sample-efficient for convolutional neural networks","authors":"Mohammed Nowaz, Rabbani Chowdhury, Shuai Zhang, Meng Wang, Sijia Liu, Pin-Yu Chen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Clark et al., 2022] Aidan Clark, Diego De Las Casas, Aure- lia Guy, Arthur Mensch, Michela Paganini, Jordan Hoff- mann, Bogdan Damoc, Blake Hechtman, Trevor Cai, Se- bastian Borgeaud, et al. Unified scaling laws for routed language models. In International Conference on Machine Learning, pages 4057-4086. PMLR, 2022.","title":"Unified scaling laws for routed language models","authors":"Aidan Clark, Diego De, Las Casas, Aurelia Guy, Arthur Mensch, Michela Paganini, Jordan Hoffmann, Bogdan Damoc, Blake Hechtman, Trevor Cai, Sebastian Borgeaud","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dai et al., 2022] Damai Dai, Li Dong, Shuming Ma, Bo Zheng, Zhifang Sui, Baobao Chang, and Furu Wei. Sta- blemoe: Stable routing strategy for mixture of experts. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7085-7095, 2022.","title":"Stablemoe: Stable routing strategy for mixture of experts","authors":"Damai Dai, Li Dong, Shuming Ma, Bo Zheng, Zhifang Sui, Baobao Chang, Furu Wei","year":"2022","venue":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fedus et al., 2022] William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parame- ter models with simple and efficient sparsity. The Journal of Machine Learning Research, 23(1):5232-5270, 2022.","title":"Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity","authors":"William Fedus, Barret Zoph, Noam Shazeer","year":"2022","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2020] Yuanyuan Guo, Yifan Xia, Jing Wang, Hui Yu, and Rung-Ching Chen. Real-time facial affective computing on mobile devices. Sensors, 20(3):870, 2020.","title":"Real-time facial affective computing on mobile devices","authors":"Yuanyuan Guo, Yifan Xia, Jing Wang, Hui Yu, Rung-Ching Chen","year":"2020","venue":"Sensors","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2021] Jiaao He, Jiezhong Qiu, Aohan Zeng, Zhilin Yang, Jidong Zhai, and Jie Tang. Fastmoe: A fast mixture-of-expert training system. arXiv preprint arXiv:2103.13262, 2021.","title":"Fastmoe: A fast mixture-of-expert training system","authors":"He","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jacobs et al., 1991] Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mix- tures of local experts. Neural computation, 3(1):79-87, 1991.","title":"Adaptive mixtures of local experts","authors":"Robert A Jacobs, Michael I Jordan, Steven J Nowlan, Geoffrey E Hinton","year":"1991","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kenton and Toutanova, 2019] Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. In Proceedings of NAACL-HLT, pages 4171-4186, 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton, Lee Kristina, Toutanova","year":"2019","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kudugunta et al., 2021] Sneha Kudugunta, Yanping Huang, Ankur Bapna, Maxim Krikun, Dmitry Lepikhin, Minh- Thang Luong, and Orhan Firat. Beyond distillation: Task- level mixture-of-experts for efficient inference. In Find- ings of the Association for Computational Linguistics: EMNLP 2021, pages 3577-3599, 2021.","title":"Beyond distillation: Tasklevel mixture-of-experts for efficient inference","authors":"Sneha Kudugunta, Yanping Huang, Ankur Bapna, Maxim Krikun, Dmitry Lepikhin, Minh-Thang Luong, Orhan Firat","year":"2021","venue":"Findings of the Association for Computational Linguistics: EMNLP 2021","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kumar et al., 2022] N Ashok Kumar, A Kavitha, P Venka- tramana, and Durgesh Nandan. Architecture design: Network-on-chip. In VLSI Architecture for Signal, Speech, and Image Processing, pages 147-165. Apple Academic Press, 2022.","title":"Architecture design: Network-on-chip","authors":"Ashok Kumar, A Kavitha, P Venkatramana, Durgesh Nandan","year":"2022","venue":"VLSI Architecture for Signal, Speech, and Image Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lepikhin et al., 2020] Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. Gshard: Scaling giant models with conditional computa- tion and automatic sharding. In International Conference on Learning Representations, 2020.","title":"Gshard: Scaling giant models with conditional computation and automatic sharding","authors":"Dmitry Lepikhin, Hyoukjoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, Zhifeng Chen","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lewis et al., 2021] Mike Lewis, Shruti Bhosale, Tim Dettmers, Naman Goyal, and Luke Zettlemoyer. Base lay- ers: Simplifying training of large, sparse models. In Inter- national Conference on Machine Learning, pages 6265- 6274. PMLR, 2021.","title":"Base layers: Simplifying training of large, sparse models","authors":"Mike Lewis, Shruti Bhosale, Tim Dettmers, Naman Goyal, Luke Zettlemoyer","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2022a] Jianjun Li, Yu Han, Ming Zhang, Gang Li, and Baohua Zhang. Multi-scale residual network model combined with global average pooling for action recog- nition. Multimedia Tools and Applications, pages 1-19, 2022.","title":"Multi-scale residual network model combined with global average pooling for action recognition","authors":"Li","year":"2022","venue":"Multimedia Tools and Applications","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2022b] Yuan Li, Ke Wang, Hao Zheng, Ahmed Louri, and Avinash Karanth. Ascend: A scalable and energy-efficient deep neural network accelerator with pho- tonic interconnects. IEEE Transactions on Circuits and Systems I: Regular Papers, 69(7):2730-2741, 2022.","title":"Ascend: A scalable and energy-efficient deep neural network accelerator with photonic interconnects","authors":"Li","year":"2022","venue":"IEEE Transactions on Circuits and Systems I: Regular Papers","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liao et al., 2021] Heng Liao, Jiajin Tu, Jing Xia, Hu Liu, Xiping Zhou, Honghui Yuan, and Yuxing Hu. As- cend: a scalable and unified architecture for ubiqui- tous deep neural network computing: Industry track pa- per. In 2021 IEEE International Symposium on High- Performance Computer Architecture (HPCA), pages 789- 801. IEEE, 2021.","title":"Ascend: a scalable and unified architecture for ubiquitous deep neural network computing: Industry track paper","authors":"Heng Liao, Jiajin Tu, Jing Xia, Hu Liu, Xiping Zhou, Honghui Yuan, Yuxing Hu","year":"2021","venue":"IEEE International Symposium on High-Performance Computer Architecture (HPCA)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mi et al., 2022] Fei Mi, Yitong Li, Yulong Zeng, Jingyan Zhou, Yasheng Wang, Chuanfei Xu, Lifeng Shang, Xin Jiang, Shiqi Zhao, and Qun Liu. Pangu-bot: Efficient generative dialogue pre-training from pre-trained language model. arXiv preprint arXiv:2203.17090, 2022.","title":"Pangu-bot: Efficient generative dialogue pre-training from pre-trained language model","authors":"Fei Mi, Yitong Li, Yulong Zeng, Jingyan Zhou, Yasheng Wang, Chuanfei Xu, Lifeng Shang, Xin Jiang, Shiqi Zhao, Qun Liu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nie et al., 2022] Xiaonan Nie, Pinxue Zhao, Xupeng Miao, Tong Zhao, and Bin Cui. Hetumoe: An efficient trillion- scale mixture-of-expert distributed training system. arXiv preprint arXiv:2203.14685, 2022.","title":"Hetumoe: An efficient trillionscale mixture-of-expert distributed training system","authors":"Xiaonan Nie, Pinxue Zhao, Xupeng Miao, Tong Zhao, Bin Cui","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nie et al., 2023] Xiaonan Nie, Xupeng Miao, Zilong Wang, Zichao Yang, Jilong Xue, Lingxiao Ma, Gang Cao, and Bin Cui. Flexmoe: Scaling large-scale sparse pre-trained model training via dynamic device placement. Proceed- ings of the ACM on Management of Data, 1(1):1-19, 2023.","title":"Flexmoe: Scaling large-scale sparse pre-trained model training via dynamic device placement","authors":"Xiaonan Nie, Xupeng Miao, Zilong Wang, Zichao Yang, Jilong Xue, Lingxiao Ma, Gang Cao, Bin Cui","year":"2023","venue":"Proceedings of the ACM on Management of Data","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Puigcerver et al., 2020] Joan Puigcerver, Carlos Riquelme Ruiz, Basil Mustafa, Cedric Renggli, Andr 'e Susano Pinto, Sylvain Gelly, Daniel Keysers, and Neil Houlsby. Scalable transfer learning with expert models. In Interna- tional Conference on Learning Representations, 2020.","title":"Scalable transfer learning with expert models","authors":"Joan Puigcerver, Carlos Riquelme Ruiz, Basil Mustafa, Cedric Renggli, ' Andr, Susano Pinto, Sylvain Gelly, Daniel Keysers, Neil Houlsby","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rajbhandari et al., 2022] Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, and Yuxiong He. Deepspeed-moe: Advancing mixture-of-experts inference and training to power next-generation ai scale. In Inter- national Conference on Machine Learning, pages 18332- 18346. PMLR, 2022.","title":"Deepspeed-moe: Advancing mixture-of-experts inference and training to power next-generation ai scale","authors":"Rajbhandari","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ren et al., 2023] Xiaozhe Ren, Pingyi Zhou, Xinfan Meng, Xinjing Huang, Yadao Wang, Weichao Wang, Pengfei Li, Xiaoda Zhang, Alexander Podolskiy, Grigory Arshinov, et al. Pangu-σ: Towards trillion parameter language model with sparse heterogeneous computing. arXiv e-prints, pages arXiv-2303, 2023.","title":"Pangu-σ: Towards trillion parameter language model with sparse heterogeneous computing","authors":"Ren","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Roller et al., 2021] Stephen Roller, Sainbayar Sukhbaatar, Jason Weston, et al. Hash layers for large sparse mod- els. Advances in Neural Information Processing Systems, 34:17555-17566, 2021.","title":"Hash layers for large sparse models","authors":"Stephen Roller, Sainbayar Sukhbaatar, Jason Weston","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Shazeer et al., 2016] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hin- ton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In Interna- tional Conference on Learning Representations, 2016.","title":"Outrageously large neural networks: The sparsely-gated mixture-of-experts layer","authors":"Shazeer","year":"2016","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Shen et al., 2023] Bo Shen, Jiaxin Zhang, Taihong Chen, Daoguang Zan, Bing Geng, An Fu, Muhan Zeng, Ailun Yu, Jichuan Ji, Jingyang Zhao, et al. Pangu-coder2: Boost- ing large language models for code with ranking feedback. arXiv preprint arXiv:2307.14936, 2023.","title":"Pangu-coder2: Boosting large language models for code with ranking feedback","authors":"Bo Shen, Jiaxin Zhang, Taihong Chen, Daoguang Zan, Bing Geng, An Fu, Muhan Zeng, Ailun Yu, Jichuan Ji, Jingyang Zhao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tong et al., 2021] Zhihao Tong, Ning Du, Xiaobo Song, and Xiaoli Wang. Study on mindspore deep learn- ing framework. In 2021 17th International Conference on Computational Intelligence and Security (CIS), pages 183-186. IEEE, 2021.","title":"Study on mindspore deep learning framework","authors":"Zhihao Tong, Ning Du, Xiaobo Song, Xiaoli Wang","year":"2021","venue":"17th International Conference on Computational Intelligence and Security (CIS)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Touvron et al., 2023] Hugo Touvron, Thibaut Lavril, Gau- tier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- othée Lacroix, Baptiste Rozière, Naman Goyal, Eric Ham- bro, Faisal Azhar, et al. Llama: Open and efficient founda- tion language models. arXiv preprint arXiv:2302.13971, 2023.","title":"Hugo Touvron, Thibaut Lavril, Gautier Izacard","authors":"Touvron","year":"2023","venue":"Llama: Open and efficient founda-tion language models","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2023] Yunhe Wang, Hanting Chen, Yehui Tang, Tianyu Guo, Kai Han, Ying Nie, Xutao Wang, Hailin Hu, Zheyuan Bai, Yun Wang, et al. Pangu-π: Enhancing language model architectures via nonlinearity compensation. arXiv preprint arXiv:2312.17276, 2023.","title":"Pangu-π: Enhancing language model architectures via nonlinearity compensation","authors":"Yunhe Wang, Hanting Chen, Yehui Tang, Tianyu Guo, Kai Han, Ying Nie, Xutao Wang, Hailin Hu, Zheyuan Bai, Yun Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xia et al., 2021] Jing Xia, Chuanning Cheng, Xiping Zhou, Yuxing Hu, and Peter Chun. Kunpeng 920: The first 7-nm chiplet-based 64-core arm soc for cloud services. IEEE Micro, 41(5):67-75, 2021.","title":"Kunpeng 920: The first 7-nm chiplet-based 64-core arm soc for cloud services","authors":"Jing Xia, Chuanning Cheng, Xiping Zhou, Yuxing Hu, Peter Chun","year":"2021","venue":"IEEE Micro","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zeng et al., 2021] Wei Zeng, Xiaozhe Ren, Teng Su, Hui Wang, Yi Liao, Zhiwei Wang, Xin Jiang, ZhenZhang Yang, Kaisheng Wang, Xiaoda Zhang, et al. Pangu- α: Large-scale autoregressive pretrained chinese language models with auto-parallel computation. arXiv preprint arXiv:2104.12369, 2021.","title":"Panguα: Large-scale autoregressive pretrained chinese language models with auto-parallel computation","authors":"Wei Zeng, Xiaozhe Ren, Teng Su, Hui Wang, Yi Liao, Zhiwei Wang, Xin Jiang, Zhenzhang Yang, Kaisheng Wang, Xiaoda Zhang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zuo et al., 2021] Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan, Ruofei Zhang, Jianfeng Gao, and Tuo Zhao. Taming sparsely activated transformer with stochastic experts. In International Conference on Learning Representations, 2021.","title":"Taming sparsely activated transformer with stochastic experts","authors":"Zuo","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}