{"type":"ScholarlyArticle","title":"MultiQuant: Training Once for Multi-bit Quantization of Neural Networks","authors":[{"name":"Ke Xu","slug":"ke-xu","given":"Ke","family":"Xu","affiliation":"School of Artificial Intelligence , Anhui University , Hefei , China; Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education , Hefei , China"},{"name":"Qiantai Feng","slug":"qiantai-feng","given":"Qiantai","family":"Feng","affiliation":"Institute of Information Science , Beijing Jiaotong University , Beijing , China; Beijing Key Laboratory of Advanced Information Science and Network Technology , Beijing , China"},{"name":"Xingyi Zhang","slug":"xingyi-zhang","given":"Xingyi","family":"Zhang","affiliation":"School of Artificial Intelligence , Anhui University , Hefei , China; Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education , Hefei , China"},{"name":"Dong Wang","slug":"dong-wang","given":"Dong","family":"Wang","affiliation":"Institute of Information Science , Beijing Jiaotong University , Beijing , China; Beijing Key Laboratory of Advanced Information Science and Network Technology , Beijing , China"}],"abstract":"Quantization has become a popular technique to compress deep neural networks (DNNs) and reduce computational costs, but most prior work focuses on training DNNs at each individual fixed bit-width and accuracy trade-off point. How to produce a model with flexible precision is largely unexplored. This work proposes a multi-bit quantization framework (MultiQuant) to make the learned DNNs robust for different precision configuration during inference by adopting Lowest-Random-Highest bit-width co-training method. Meanwhile, we propose an online adaptive label generation strategy to alleviate the problem of vicious competition under different precision caused by one-hot labels in the supernet training. The trained supernet model can be flexibly set to different bit widths to support dynamic speed and accuracy trade-off. Furthermore, we adopt the Monte Carlo sampling-based genetic algorithm search strategy with quantization-aware accuracy predictor as evaluation criterion to incorporate the mixed precision technology in our framework. Experiment results on ImageNet datasets demonstrate MultiQuant method can attain the quantization results under different bit-widths comparable with quantization-aware training without retraining.","doi":"10.24963/ijcai.2022/504","url":"https://ijcai.org/proceedings/2022/504","pdf":"https://ijcai.org/proceedings/2022/0504.pdf","markdown":"https://ijcai.org/proceedings/2022/504.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/504","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":504,"pages":{"first":3629,"last":3635},"keywords":["Machine Learning: Convolutional Networks","Machine Learning: Automated Machine Learning","Machine Learning: Classification","Machine Learning: Robustness"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986716","length_sec":70,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984859","length_sec":319,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Alizadeh et al., 2020] Milad Alizadeh, Arash Behboodi, Mart van Baalen, Christos Louizos, Tijmen Blankevoort, and Max Welling. Gradient ℓ 1 regularization for quantiza- tion robustness. In Proc. of ICLR, 2020.","title":"Gradient ℓ 1 regularization for quantization robustness","authors":"Milad Alizadeh, Arash Behboodi, Mart Van Baalen, Christos Louizos, Tijmen Blankevoort, Max Welling","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai et al., 2020] Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. Once-for-all: Train one network and specialize it for efficient deployment. In Proc. of ICLR, 2020.","title":"Once-for-all: Train one network and specialize it for efficient deployment","authors":"Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, Song Han","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Choi et al., 2018] Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srini- vasan, and Kailash Gopalakrishnan. Pact: Parameterized clipping activation for quantized neural networks. ArXiv preprint, 2018.","title":"Pact: Parameterized clipping activation for quantized neural networks","authors":"Choi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Fei-Fei Li. Imagenet: A large-scale hierarchical image database. In Proc. of CVPR, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Fei-Fei Li","year":"2009","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Esser et al., 2020] Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dhar- mendra S. Modha. Learned step size quantization. In Proc. of ICLR, 2020.","title":"Learned step size quantization","authors":"K Steven, Jeffrey L Esser, Deepika Mckinstry, Rathinakumar Bablani, Dharmendra S Appuswamy, Modha","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gholami et al., 2021] Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural net- work inference. ArXiv preprint, 2021.","title":"A survey of quantization methods for efficient neural network inference","authors":"Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, Kurt Keutzer","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gong et al., 2019] Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In Proc. of ICCV, 2019.","title":"Differentiable soft quantization: Bridging full-precision and low-bit neural networks","authors":"Gong","year":"2019","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proc. of CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proc. of ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2021] Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. BRECQ: pushing the limit of post-training quantization by block reconstruction. In Proc. of ICLR, 2021.","title":"BRECQ: pushing the limit of post-training quantization by block reconstruction","authors":"Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, Shi Gu","year":"2021","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Maaten and Hinton, 2008] L. V. D. Maaten and Geoffrey E. Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 2008.","title":"Visualizing data using t-sne","authors":"L V D Maaten, Geoffrey E Hinton","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Migacz, 2017] Szymon Migacz. 8-bit inference with tensorrt. In GPU technology conference, 2017.","title":"Szymon Migacz. 8-bit inference with tensorrt","authors":"Migacz","year":"2017","venue":"GPU technology conference","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nagel et al., 2020] Markus Nagel, Rana Ali Amjad, Mart van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? adaptive rounding for post-training quantization. In Proc. of ICML, 2020.","title":"Up or down? adaptive rounding for post-training quantization","authors":"Nagel","year":"2020","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nahshan et al., 2019] Yury Nahshan, Brian Chmiel, Chaim Baskin, Evgenii Zheltonozhskii, Ron Banner, Alexander M. Bronstein, and Avi Mendelson. Loss aware post-training quantization. ArXiv preprint, 2019.","title":"Loss aware post-training quantization","authors":"Yury Nahshan, Brian Chmiel, Chaim Baskin, Evgenii Zheltonozhskii, Ron Banner, Alexander M Bronstein, Avi Mendelson","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Qin et al., 2020] Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, and Nicu Sebe. Binary neural networks: A survey. Pattern Recognition, 2020.","title":"Binary neural networks: A survey","authors":"Qin","year":"2020","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sandler et al., 2018] Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proc. of CVPR, 2018.","title":"Mobilenetv2: Inverted residuals and linear bottlenecks","authors":"Mark Sandler, Andrew G Howard, Menglong Zhu, Andrey Zhmoginov, Liang-Chieh Chen","year":"2018","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shen et al., 2021] Mingzhu Shen, Feng Liang, Ruihao Gong, Yuhang Li, Chuming Li, Chen Lin, Fengwei Yu, Junjie Yan, and Wanli Ouyang. Once quantization-aware training: High performance extremely low-bit architecture search. In Proc. of ICCV), 2021.","title":"Once quantization-aware training: High performance extremely low-bit architecture search","authors":"Mingzhu Shen, Feng Liang, Ruihao Gong, Yuhang Li, Chuming Li, Chen Lin, Fengwei Yu, Junjie Yan, Wanli Ouyang","year":"2021","venue":"Proc. of ICCV)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shkolnik et al., 2020] Moran Shkolnik, Brian Chmiel, Ron Banner, Gil Shomron, Yury Nahshan, Alex M. Bronstein, and Uri C. Weiser. Robust quantization: One model to rule them all. In Proc. of NeuIPS, 2020.","title":"Robust quantization: One model to rule them all","authors":"Moran Shkolnik, Brian Chmiel, Ron Banner, Gil Shomron, Yury Nahshan, Alex M Bronstein, Uri C Weiser","year":"2020","venue":"Proc. of NeuIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sun et al., 2021] Ximeng Sun, Rameswar Panda, Chun-Fu Chen, Naigang Wang, Bowen Pan, Kailash Gopalakrishnan, Aude Oliva, Rogério Feris, and Kate Saenko. All at once network quantization via collaborative knowledge transfer. ArXiv preprint, 2021.","title":"All at once network quantization via collaborative knowledge transfer","authors":"Sun","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Szegedy et al., 2016] Christian Szegedy, Vincent Van- houcke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wo- jna. Rethinking the inception architecture for computer vision. In Proc. of CVPR, 2016.","title":"Rethinking the inception architecture for computer vision","authors":"Szegedy","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2020a] Dilin Wang, Meng Li, Chengyue Gong, and Vikas Chandra. Attentivenas: Improving neural ar- chitecture search via attentive sampling. ArXiv preprint, 2020.","title":"Attentivenas: Improving neural architecture search via attentive sampling","authors":"Wang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2020b] Tianzhe Wang, Kuan Wang, Han Cai, Ji Lin, Zhijian Liu, Hanrui Wang, Yujun Lin, and Song Han. APQ: joint search for network architecture, pruning and quantization policy. In Proc. of CVPR, 2020.","title":"APQ: joint search for network architecture, pruning and quantization policy","authors":"Wang","year":"2020","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Whitley, 1994] Darrell Whitley. A genetic algorithm tutorial. Statistics and computing, 1994.","title":"A genetic algorithm tutorial","authors":"Darrell Whitley","year":"1994","venue":"Statistics and computing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yu et al., 2020] Jiahui Yu, Pengchong Jin, Hanxiao Liu, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Thomas S. Huang, Xiaodan Song, Ruoming Pang, and Quoc Le. Bignas: Scaling up neural architecture search with big single-stage models. In Proc. of ECCV, 2020.","title":"Bignas: Scaling up neural architecture search with big single-stage models","authors":"Jiahui Yu, Pengchong Jin, Hanxiao Liu, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Thomas S Huang, Xiaodan Song, Ruoming Pang, Quoc Le","year":"2020","venue":"Proc. of ECCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yu et al., 2021] Haichao Yu, Haoxiang Li, Honghui Shi, Thomas S. Huang, and Gang Hua. Any-precision deep neural networks. In Proc. of AAAI, 2021.","title":"Any-precision deep neural networks","authors":"Yu","year":"2021","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2016] Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefa-net: Train- ing low bitwidth convolutional neural networks with low bitwidth gradients. ArXiv preprint, 2016.","title":"Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients","authors":"Zhou","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}