{"type":"ScholarlyArticle","title":"Efficient and Modularized Training on FPGA for Real-time Applications","authors":[{"name":"Shreyas Kolala Venkataramanaiah","slug":"shreyas-kolala-venkataramanaiah","given":"Shreyas Kolala","family":"Venkataramanaiah","affiliation":"School of ECEE , Arizona State University , Tempe , AZ , USA"},{"name":"Xiaocong Du","slug":"xiaocong-du","given":"Xiaocong","family":"Du","affiliation":"School of ECEE , Arizona State University , Tempe , AZ , USA"},{"name":"Zheng Li","slug":"zheng-li","given":"Zheng","family":"Li","affiliation":"School of CIDSE , Arizona State University , Tempe , AZ , US"},{"name":"Shihui Yin","slug":"shihui-yin","given":"Shihui","family":"Yin","affiliation":"School of ECEE , Arizona State University , Tempe , AZ , USA"},{"name":"Yu Cao","slug":"yu-cao","given":"Yu","family":"Cao","affiliation":"School of ECEE , Arizona State University , Tempe , AZ , USA; School of CIDSE , Arizona State University , Tempe , AZ , US"},{"name":"Jae-sun Seo","slug":"jae-sun-seo","given":"Jae-sun","family":"Seo","affiliation":"School of ECEE , Arizona State University , Tempe , AZ , USA"}],"abstract":"Training of deep Convolution Neural Networks (CNNs) requires a tremendous amount of computation and memory and thus, GPUs are widely used to meet the computation demands of these complex training tasks. However, lacking the flexibility to exploit architectural optimizations, GPUs have poor energy efficiency of GPUs and are hard to be deployed on energy-constrained platforms. FPGAs are highly suitable for training, such as real-time learning at the edge, as they provide higher energy efficiency and better flexibility to support algorithmic evolution. This paper first develops a training accelerator on FPGA, with 16-bit fixed-point computing and various training modules. Furthermore, leveraging model segmentation techniques from Progressive Segmented Training, the newly developed FPGA accelerator is applied to online learning, achieving much lower computation cost. We demonstrate the performance of representative CNNs trained for CIFAR-10 on Intel Stratix-10 MX FPGA, evaluating both the conventional training procedure and the online learning algorithm.","doi":"10.24963/ijcai.2020/755","url":"https://ijcai.org/proceedings/2020/755","pdf":"https://ijcai.org/proceedings/2020/0755.pdf","markdown":"https://ijcai.org/proceedings/2020/755.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/755","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Demos","track":"Demos","number":755,"pages":{"first":5237,"last":5239},"keywords":["Machine Learning: general","Computer Vision: general"],"videos":[],"references":[{"position":1,"raw":"[Chaudhry et al., 2018] Arslan Chaudhry, Marc'Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00420, 2018.","title":"Efficient lifelong learning with a-gem","authors":"Arslan Chaudhry, Marc'aurelio Ranzato, Marcus Rohrbach, Mohamed Elhoseiny","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Choi et al., 2018] Seungkyu Choi, Jaehyeong Sim, Myeonggu Kang, and Lee-Sup Kim. TrainWare: A memory optimized weight update architecture for on-device convolutional neural network training. In Proceedings of the International Symposium on Low Power Electronics and Design (ISLPED), 2018.","title":"TrainWare: A memory optimized weight update architecture for on-device convolutional neural network training","authors":"Choi","year":"2018","venue":"Proceedings of the International Symposium on Low Power Electronics and Design (ISLPED)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deo et al., 2016] Manish Deo, Jeffrey Schulz, and Lance Brown. Intel stratix 10 mx devices solve the memory band- width challenge. Intel White Paper, 2016.","title":"Intel stratix 10 mx devices solve the memory bandwidth challenge","authors":"Manish Deo, Jeffrey Schulz, Lance Brown","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Du et al., 2019a] Xiaocong Du, Gouranga Charan, Frank Liu, and Yu Cao. Single-net continual learning with pro- gressive segmented training. In 2019 18th IEEE Interna- tional Conference On Machine Learning And Applications (ICMLA), pages 1629-1636, Dec 2019.","title":"Single-net continual learning with progressive segmented training","authors":"Du","year":"2019","venue":"18th IEEE International Conference On Machine Learning And Applications (ICMLA)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Du et al., 2019b] Xiaocong Du, Zheng Li, Yufei Ma, and Yu Cao. Efficient network construction through structural plasticity. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 9(3):453-464, 2019.","title":"Efficient network construction through structural plasticity","authors":"Du","year":"2019","venue":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gomperts et al., 2011] Alexander Gomperts, Abhisek Ukil, and Franz Zurfluh. Development and implementation of parameterized FPGA-based general purpose neural net- works for online applications. IEEE Transactions on In- dustrial Informatics, 7(1):78-89, 2011.","title":"Development and implementation of parameterized FPGA-based general purpose neural networks for online applications","authors":"Alexander Gomperts, Abhisek Ukil, Franz Zurfluh","year":"2011","venue":"IEEE Transactions on Industrial Informatics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Guo et al., 2019] Kaiyuan Guo, Shuang Liang, Jincheng Yu, Xuefei Ning, Wenshuo Li, Yu Wang, and Huazhong Yang. Compressed cnn training with fpga-based accelera- tor. In Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, pages 189-189, 2019.","title":"Compressed cnn training with fpga-based accelerator","authors":"Kaiyuan Guo, Shuang Liang, Jincheng Yu, Xuefei Ning, Wenshuo Li, Yu Wang, Huazhong Yang","year":"2019","venue":"Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Han et al., 2015] Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural net- works with pruning, trained quantization and huffman cod- ing. arXiv preprint arXiv:1510.00149, 2015.","title":"Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding","authors":"Han","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Han et al., 2016] Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A Horowitz, and William J Dally. Eie: efficient inference engine on compressed deep neural network. ACM SIGARCH Computer Architecture News, 44(3):243-254, 2016.","title":"Eie: efficient inference engine on compressed deep neural network","authors":"Han","year":"2016","venue":"ACM SIGARCH Computer Architecture News","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kirkpatrick et al., 2017] James Kirkpatrick, Razvan Pas- canu, Neil Rabinowitz, Joel Veness, Guillaume Des- jardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcom- ing catastrophic forgetting in neural networks. Proceed- ings of the national academy of sciences, 114(13):3521- 3526, 2017.","title":"Overcoming catastrophic forgetting in neural networks","authors":"James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska","year":"2017","venue":"Proceedings of the national academy of sciences","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- ages. 2009.","title":"Learning multiple layers of features from tiny images","authors":"Krizhevsky","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Hoiem, 2017] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-2947, 2017.","title":"Learning without forgetting","authors":"Zhizhong Li, Derek Hoiem","year":"2017","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2015] Zheng Li, Chenchen Liu, Yandan Wang, Bo- nan Yan, Chaofei Yang, Jianlei Yang, and Hai Li. An overview on memristor crossabr based neuromorphic cir- cuit and architecture. In 2015 IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SoC), pages 52-56. IEEE, 2015.","title":"An overview on memristor crossabr based neuromorphic circuit and architecture","authors":"Zheng Li, Chenchen Liu, Yandan Wang, Bonan Yan, Chaofei Yang, Jianlei Yang, Hai Li","year":"2015","venue":"IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SoC)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2015] Chenchen Liu, Bonan Yan, Chaofei Yang, Linghao Song, Zheng Li, Beiye Liu, Yiran Chen, Hai Li, Qing Wu, and Hao Jiang. A spiking neuromorphic design with resistive crossbar. In 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC), pages 1-6. IEEE, 2015.","title":"A spiking neuromorphic design with resistive crossbar","authors":"Chenchen Liu, Bonan Yan, Chaofei Yang, Linghao Song, Zheng Li, Beiye Liu, Yiran Chen, Hai Li, Qing Wu, Hao Jiang","year":"2015","venue":"52nd ACM/EDAC/IEEE Design Automation Conference (DAC)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2017] Zhiqiang Liu, Yong Dou, Jingfei Jiang, Qiang Wang, and Paul Chow. An FPGA-based processor for training convolutional neural networks. In Proceedings of the International Conference on Field Programmable Technology (ICFPT), pages 207-210, 2017.","title":"An FPGA-based processor for training convolutional neural networks","authors":"Zhiqiang Liu, Yong Dou, Jingfei Jiang, Qiang Wang, Paul Chow","year":"2017","venue":"Proceedings of the International Conference on Field Programmable Technology (ICFPT)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2018] Qiang Liu, Jia Liu, Ruoyu Sang, Jiajun Li, Tao Zhang, and Qijun Zhang. Fast neural network training on FPGA using quasi-newton optimization method. IEEE Transactions on Very Large Scale Integration (VLSI) Sys- tems, 26(8):1575-1579, 2018.","title":"Fast neural network training on FPGA using quasi-newton optimization method","authors":"Qiang Liu, Jia Liu, Ruoyu Sang, Jiajun Li, Tao Zhang, Qijun Zhang","year":"2018","venue":"IEEE Transactions on Very Large Scale Integration (VLSI) Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rafael et al., 2005] G Rafael, C Ricardo, C Joaquín, C An- gel, and Wakamura M Maeda. FPGA implementation of a pipelined on-line backpropagation. Journal of VLSI Signal Processing, 40(2):189-213, 2005.","title":"FPGA implementation of a pipelined on-line backpropagation","authors":"C Rafael, C Ricardo, C Joaquín, Wakamura M Angel, Maeda","year":"2005","venue":"Journal of VLSI Signal Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Rebuffi et al., 2017] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Pro- ceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001-2010, 2017.","title":"icarl: Incremental classifier and representation learning","authors":"Sylvestre-Alvise, Alexander Rebuffi, Georg Kolesnikov, Christoph H Sperl, Lampert","year":"2017","venue":"Proceedings of the IEEE conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Venkataramanaiah et al., 2019] Shreyas Kolala Venkatara- manaiah, Yufei Ma, Shihui Yin, Eriko Nurvithadhi, Ar- avind Dasu, Yu Cao, and Jae-sun Seo. Automatic compiler based fpga accelerator for cnn training. In 2019 29th In- ternational Conference on Field Programmable Logic and Applications (FPL), pages 166-172. IEEE, 2019.","title":"Automatic compiler based fpga accelerator for cnn training","authors":"Kolala Shreyas, Yufei Venkataramanaiah, Shihui Ma, Eriko Yin, Aravind Nurvithadhi, Yu Dasu, Jae-Sun Cao, Seo","year":"2019","venue":"29th International Conference on Field Programmable Logic and Applications (FPL)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhao et al., 2016] Wenlai Zhao, Haohuan Fu, Wayne Luk, Teng Yu, Shaojun Wang, Bo Feng, Yuchun Ma, and Guangwen Yang. F-CNN: An FPGA-based framework for training convolutional neural networks. In Proceed- ings of the IEEE International Conference on Application- specific Systems, Architectures and Processors (ASAP), pages 107-114, 2016.","title":"F-CNN: An FPGA-based framework for training convolutional neural networks","authors":"Zhao","year":"2016","venue":"Proceedings of the IEEE International Conference on Applicationspecific Systems, Architectures and Processors (ASAP)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}