{"type":"ScholarlyArticle","title":"HyQ: Hardware-Friendly Post-Training Quantization for CNN-Transformer Hybrid Networks","authors":[{"name":"Nam Joon Kim","slug":"nam-joon-kim","given":"Nam Joon","family":"Kim","affiliation":"Department of Electrical and Information Engineering and Research Center for Electrical and Information Technology , Seoul National University of Science and Technology"},{"name":"Jongho Lee","slug":"jongho-lee","given":"Jongho","family":"Lee","affiliation":"Department of Electrical and Information Engineering and Research Center for Electrical and Information Technology , Seoul National University of Science and Technology; SqueezeBits Inc."},{"name":"Hyun Kim","slug":"hyun-kim","given":"Hyun","family":"Kim","affiliation":"Department of Electrical and Information Engineering and Research Center for Electrical and Information Technology , Seoul National University of Science and Technology"}],"abstract":"Hybrid models that combine CNNs and ViTs have recently emerged as state-of-the-art computer vision models. To efficiently deploy these hybrid models on resource-constrained mobile/edge devices, quantization is emerging as a promising solution. However, post-training quantization (PTQ), which does not require retraining or labeled data, has not been extensively studied for hybrid models. In this study, we propose a novel PTQ technique specialized for CNN-transformer hybrid models by considering the hardware design of hybrid models on AI accelerators such as GPUs and FPGAs. First, we introduce quantization-aware distribution scaling to address the large outliers caused by inter-channel variance in convolution layers. Furthermore, in the transformer block, we propose approximating the integer-only softmax with a linear function. This approach allows us to avoid costly FP32/INT32 multiplications, resulting in more efficient computations. Experimental results show that the proposed quantization method with INT8 precision demonstrated a 0.39% accuracy drop compared with the FP32 baseline on MobileViT-s with the ImageNet-1k dataset. Furthermore, when implemented on the FPGA platform, the proposed linear softmax achieved significant resource savings, reducing the look-up table and flip-flop usage by 1.8 ~ 2.1x and 1.3 ~ 1.9x, respectively, compared with the existing second-order polynomial approximation. The code is available at https://github.com/IDSL-SeoulTech/HyQ.","doi":"10.24963/ijcai.2024/474","url":"https://ijcai.org/proceedings/2024/474","pdf":"https://ijcai.org/proceedings/2024/0474.pdf","markdown":"https://ijcai.org/proceedings/2024/474.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/474","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":474,"pages":{"first":4291,"last":4299},"keywords":["Machine Learning: ML: Deep learning architectures","Machine Learning: ML: Optimization","Computer Vision: CV: Recognition (object detection, categorization)","Computer Vision: CV: Machine learning for vision"],"videos":[],"references":[{"position":1,"raw":"[Cai et al., 2019] Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. Once-for-all: Train one network and specialize it for efficient deployment. arXiv preprint arXiv:1908.09791, 2019.","title":"Once-for-all: Train one network and specialize it for efficient deployment","authors":"Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, Song Han","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Choukroun et al., 2019] Yoni Choukroun, Eli Kravchik, Fan Yang, and Pavel Kisilev. Low-bit quantization of neu- ral networks for efficient inference. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pages 3009-3018. IEEE, 2019.","title":"Low-bit quantization of neural networks for efficient inference","authors":"Yoni Choukroun, Eli Kravchik, Fan Yang, Pavel Kisilev","year":"2019","venue":"IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gysel et al., 2018] Philipp Gysel, Jon Pimentel, Moham- mad Motamedi, and Soheil Ghiasi. Ristretto: A framework for empirical study of resource-efficient inference in con- volutional neural networks. IEEE transactions on neural networks and learning systems, 29(11):5784-5789, 2018.","title":"Ristretto: A framework for empirical study of resource-efficient inference in convolutional neural networks","authors":"Philipp Gysel, Jon Pimentel, Mohammad Motamedi, Soheil Ghiasi","year":"2018","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hendrycks and Gimpel, 2016a] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016.","title":"Gaussian error linear units","authors":"Gimpel Hendrycks, Dan Hendrycks, Kevin Gimpel","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hendrycks and Gimpel, 2016b] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016.","title":"Gaussian error linear units","authors":"Gimpel Hendrycks, Dan Hendrycks, Kevin Gimpel","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jacob et al., 2018] Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2704-2713, 2018.","title":"Quantization and training of neural networks for efficient integer-arithmetic-only inference","authors":"Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, Dmitry Kalenichenko","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim and Kim, 2021] Sungrae Kim and Hyun Kim. Zero- centered fixed-point quantization with iterative retraining for deep convolutional neural network-based object detec- tors. IEEE Access, 9:20828-20839, 2021.","title":"Zerocentered fixed-point quantization with iterative retraining for deep convolutional neural network-based object detectors","authors":"Kim ; Kim, Sungrae Kim, Hyun Kim","year":"2021","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kim and Kim, 2023] Nam Joon Kim and Hyun Kim. Fp- agl: Filter pruning with adaptive gradient learning for accelerating deep convolutional neural networks. IEEE Transactions on Multimedia, 25:5279-5290, 2023.","title":"Fpagl: Filter pruning with adaptive gradient learning for accelerating deep convolutional neural networks","authors":"Kim ; ] Kim, Joon Nam, Hyun Kim, Kim","year":"2023","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kim et al., 2021] Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. I-bert: Integer- only bert quantization. In International conference on ma- chine learning, pages 5506-5518. PMLR, 2021.","title":"I-bert: Integeronly bert quantization","authors":"Kim","year":"2021","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kulkarni et al., 2021] Uday Kulkarni, SM Meena, Sunil V Gurlahosur, and Gopal Bhogar. Quantization friendly mo- bilenet (qf-mobilenet) architecture for vision based ap- plications on embedded platforms. Neural Networks, 136:28-39, 2021.","title":"Quantization friendly mobilenet (qf-mobilenet) architecture for vision based applications on embedded platforms","authors":"Uday Kulkarni, Meena, Gopal Sunil V Gurlahosur, Bhogar","year":"2021","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lee et al., 2023] Jemin Lee, Yongin Kwon, Jeman Park, Misun Yu, and Hwanjun Song. Q-hyvit: Post-training quantization for hybrid vision transformer with bridge block reconstruction. arXiv preprint arXiv:2303.12557, 2023.","title":"Q-hyvit: Post-training quantization for hybrid vision transformer with bridge block reconstruction","authors":"Lee","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li and Gu, 2022] Zhikai Li and Qingyi Gu. I-vit: integer- only quantization for efficient vision transformer infer- ence. arXiv preprint arXiv:2207.01405, 2022.","title":"I-vit: integeronly quantization for efficient vision transformer inference","authors":"Li, Gu, Zhikai Li, Qingyi Gu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2019] Rundong Li, Yan Wang, Feng Liang, Hong- wei Qin, Junjie Yan, and Rui Fan. Fully quantized network for object detection. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 2810-2819, 2019.","title":"Fully quantized network for object detection","authors":"Rundong Li, Yan Wang, Feng Liang, Hongwei Qin, Junjie Yan, Rui Fan","year":"2019","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2022] Yanyu Li, Geng Yuan, Yang Wen, Ju Hu, Georgios Evangelidis, Sergey Tulyakov, Yanzhi Wang, and Jian Ren. Efficientformer: Vision transformers at mo- bilenet speed. Advances in Neural Information Processing Systems, 35:12934-12949, 2022.","title":"Efficientformer: Vision transformers at mobilenet speed","authors":"Yanyu Li, Geng Yuan, Yang Wen, Ju Hu, Georgios Evangelidis, Sergey Tulyakov, Yanzhi Wang, Jian Ren","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lin et al., 2021] Yang Lin, Tianyu Zhang, Peiqin Sun, Zheng Li, and Shuchang Zhou. Fq-vit: Post-training quantization for fully quantized vision transformer. arXiv preprint arXiv:2111.13824, 2021.","title":"Fq-vit: Post-training quantization for fully quantized vision transformer","authors":"Yang Lin, Tianyu Zhang, Peiqin Sun, Zheng Li, Shuchang Zhou","year":"2021","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2022/164"},{"position":20,"raw":"[Liu et al., 2021a] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF in- ternational conference on computer vision, pages 10012- 10022, 2021.","title":"Swin transformer: Hierarchical vision transformer using shifted windows","authors":"Liu","year":"2021","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2021b] Zhenhua Liu, Yunhe Wang, Kai Han, Wei Zhang, Siwei Ma, and Wen Gao. Post-training quantiza- tion for vision transformer. Advances in Neural Informa- tion Processing Systems, 34:28092-28103, 2021.","title":"Post-training quantization for vision transformer","authors":"Liu","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2022] Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12009-12019, 2022.","title":"Swin transformer v2: Scaling up capacity and resolution","authors":"Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong","year":"2022","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2023] Yijiang Liu, Huanrui Yang, Zhen Dong, Kurt Keutzer, Li Du, and Shanghang Zhang. Noisyquant: Noisy bias-enhanced post-training activation quantization for vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20321-20330, 2023.","title":"Noisyquant: Noisy bias-enhanced post-training activation quantization for vision transformers","authors":"Liu","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Mehta and Rastegari, 2021] Sachin Mehta and Mohammad Rastegari. Mobilevit: light-weight, general-purpose, and mobile-friendly vision transformer. arXiv preprint arXiv:2110.02178, 2021.","title":"Mobilevit: light-weight, general-purpose, and mobile-friendly vision transformer","authors":"Sachin Mehta, Mohammad Rastegari","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mehta and Rastegari, 2022] Sachin Mehta and Mohammad Rastegari. Separable self-attention for mobile vision trans- formers. arXiv preprint arXiv:2206.02680, 2022.","title":"Separable self-attention for mobile vision transformers","authors":"Sachin Mehta, Mohammad Rastegari","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Nagel et al., 2019] Markus Nagel, Mart van Baalen, Tijmen Blankevoort, and Max Welling. Data-free quantization through weight equalization and bias correction. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 1325-1334, 2019.","title":"Data-free quantization through weight equalization and bias correction","authors":"Markus Nagel, Mart Van Baalen, Tijmen Blankevoort, Max Welling","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Nagel et al., 2021] Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart Van Baalen, and Tijmen Blankevoort. A white paper on neural network quantization. arXiv preprint arXiv:2106.08295, 2021.","title":"A white paper on neural network quantization","authors":"Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart Van Baalen, Tijmen Blankevoort","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Naseer et al., 2021] Muhammad Muzammal Naseer, Kan- chana Ranasinghe, Salman H Khan, Munawar Hayat, Fa- had Shahbaz Khan, and Ming-Hsuan Yang. Intriguing properties of vision transformers. Advances in Neural In- formation Processing Systems, 34:23296-23308, 2021.","title":"Muhammad Muzammal Naseer, Kanchana Ranasinghe","authors":"Naseer","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Nguyen et al., 2020a] Duy Thanh Nguyen, Nguyen Huy Hung, Hyun Kim, and Hyuk-Jae Lee. An approximate memory architecture for energy saving in deep learning applications. IEEE Transactions on Circuits and Systems I: Regular Papers, 67(5):1588-1601, 2020.","title":"An approximate memory architecture for energy saving in deep learning applications","authors":"Nguyen","year":"2020","venue":"IEEE Transactions on Circuits and Systems I: Regular Papers","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Nguyen et al., 2020b] Duy Thanh Nguyen, Hyun Kim, and Hyuk-Jae Lee. Layer-specific optimization for mixed data flow with mixed precision in fpga design for cnn-based ob- ject detectors. IEEE Transactions on Circuits and Systems for Video Technology, 31(6):2450-2464, 2020.","title":"Layer-specific optimization for mixed data flow with mixed precision in fpga design for cnn-based object detectors","authors":"Nguyen","year":"2020","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural in- formation processing systems, 32, 2019.","title":"Pytorch: An imperative style, highperformance deep learning library","authors":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Rahman et al., 2016] Atul Rahman, Jongeun Lee, and Kiy- oung Choi. Efficient fpga acceleration of convolutional neural networks using logical-3d compute array. In 2016 Design, Automation & Test in Europe Conference & Exhi- bition (DATE), pages 1393-1398. IEEE, 2016.","title":"Efficient fpga acceleration of convolutional neural networks using logical-3d compute array","authors":"Atul Rahman, Jongeun Lee, Kiyoung Choi","year":"2016","venue":"2016 Design, Automation & Test in Europe Conference & Exhibition (DATE)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ramachandran et al., 2017] Prajit Ramachandran, Barret Zoph, and Quoc V Le. Searching for activation functions. arXiv preprint arXiv:1710.05941, 2017.","title":"Searching for activation functions","authors":"Ramachandran","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Sandler et al., 2018] Mark Sandler, Andrew Howard, Men- glong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510-4520, 2018.","title":"Mobilenetv2: Inverted residuals and linear bottlenecks","authors":"Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, Liang-Chieh Chen","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Steiner et al., 2021] Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How to train your vit? data, augmentation, and regularization in vision transformers. arXiv preprint arXiv:2106.10270, 2021.","title":"How to train your vit? data, augmentation, and regularization in vision transformers","authors":"Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, Lucas Beyer","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Tan and Le, 2019] Mingxing Tan and Quoc Le. Efficient- net: Rethinking model scaling for convolutional neural networks. In International conference on machine learn- ing, pages 6105-6114. PMLR, 2019.","title":"Efficientnet: Rethinking model scaling for convolutional neural networks","authors":"Le ; Tan, Mingxing Tan, Quoc Le","year":"2019","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wei et al., 2023] Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong, Jinyang Guo, and Xiang- long Liu. Outlier suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling. arXiv preprint arXiv:2304.09145, 2023.","title":"Outlier suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling","authors":"Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong, Jinyang Guo, Xianglong Liu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wightman, 2019] Ross Wightman. Pytorch image mod- els. https://github.com/rwightman/pytorch-image-models, 2019. Accessed: 2023-08-08.","title":"Pytorch image models","authors":"Ross Wightman","year":"2023","venue":null,"doi":null,"url":"https://github.com/rwightman/pytorch-image-models,2019","internal_url":null},{"position":39,"raw":"[Xiao et al., 2023] Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087-38099. PMLR, 2023.","title":"Smoothquant: Accurate and efficient post-training quantization for large language models","authors":"Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, Song Han","year":"2023","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Yang et al., 2022] Chenglin Yang, Siyuan Qiao, Qihang Yu, Xiaoding Yuan, Yukun Zhu, Alan Yuille, Hartwig Adam, and Liang-Chieh Chen. Moat: Alternating mobile con- volution and attention brings strong vision models. arXiv preprint arXiv:2210.01820, 2022.","title":"Moat: Alternating mobile convolution and attention brings strong vision models","authors":"Yang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Yin et al., 2019] Penghang Yin, Jiancheng Lyu, Shuai Zhang, Stanley Osher, Yingyong Qi, and Jack Xin. Under- standing straight-through estimator in training activation quantized neural nets. arXiv preprint arXiv:1903.05662, 2019.","title":"Understanding straight-through estimator in training activation quantized neural nets","authors":"Yin","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[You et al., 2020] Haoran You, Xiaohan Chen, Yongan Zhang, Chaojian Li, Sicheng Li, Zihao Liu, Zhangyang Wang, and Yingyan Lin. Shiftaddnet: A hardware-inspired deep network. Advances in Neural Information Processing Systems, 33:2771-2783, 2020.","title":"Shiftaddnet: A hardware-inspired deep network","authors":"Haoran You, Xiaohan Chen, Yongan Zhang, Chaojian Li, Sicheng Li, Zihao Liu, Zhangyang Wang, Yingyan Lin","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Yuan et al., 2022] Zhihang Yuan, Chenhao Xue, Yiqi Chen, Qiang Wu, and Guangyu Sun. Ptq4vit: Post-training quan- tization for vision transformers with twin uniform quan- tization. In European Conference on Computer Vision, pages 191-207. Springer, 2022.","title":"Ptq4vit: Post-training quantization for vision transformers with twin uniform quantization","authors":"Yuan","year":"2022","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhou et al., 2022] Minxuan Zhou, Weihong Xu, Jaeyoung Kang, and Tajana Rosing. Transpim: A memory-based acceleration via software-hardware co-design for trans- former. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 1071-1085. IEEE, 2022.","title":"Transpim: A memory-based acceleration via software-hardware co-design for transformer","authors":"Zhou","year":"2022","venue":"IEEE International Symposium on High-Performance Computer Architecture (HPCA)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}