{"type":"ScholarlyArticle","title":"Efficient Screen Content Image Compression via Superpixel-based Content Aggregation and Dynamic Feature Fusion","authors":[{"name":"Sheng Shen","slug":"sheng-shen","given":"Sheng","family":"Shen","affiliation":"Tianjin University"},{"name":"Huanjing Yue","slug":"huanjing-yue","given":"Huanjing","family":"Yue","affiliation":"Tianjin University"},{"name":"Jingyu Yang","slug":"jingyu-yang","given":"Jingyu","family":"Yang","affiliation":"Tianjin University"}],"abstract":"This paper addresses the challenge of efficiently compressing screen content images (SCIs) – computer generated images with unique attributes such as large uniform regions, sharp edges, and limited color palettes, which pose difficulties for conventional compression algorithms. We propose a Superpixel-based Content Aggregation Block (SCAB) to aggregate local pixels into one super-pixel and aggregate non-local information via super-pixel transformer. Such aggregation enables the dynamic assimilation of non-local information while maintaining manageable complexity. Furthermore, we enhance our channel-wise context entropy model with a Dynamic Feature Fusion (DFF) mechanism. This  mechanism integrates decoded slices and side information dynamically based on their global correlation, allowing the network to dynamically learn the optimal weights for global information usage.  Extensive experiments on three SCI datasets (SCID, CCT, and SIQAD) show our method’s superior RD performance and inference time, making it the first network comparable with the advanced VVC-SCC standard.","doi":"10.24963/ijcai.2024/134","url":"https://ijcai.org/proceedings/2024/134","pdf":"https://ijcai.org/proceedings/2024/0134.pdf","markdown":"https://ijcai.org/proceedings/2024/134.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/134","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":134,"pages":{"first":1209,"last":1217},"keywords":["Computer Vision: CV: Other","Computer Vision: CV: Computational photography","Computer Vision: CV: Image and video synthesis and generation"],"videos":[],"references":[{"position":1,"raw":"[Ahmed et al., 1974] Nasir Ahmed, T Natarajan, and Kamisetty R Rao. Discrete cosine transform. IEEE trans- actions on Computers, 100(1):90-93, 1974.","title":"Discrete cosine transform","authors":"Nasir Ahmed, T Natarajan, Kamisetty R Rao","year":"1974","venue":"IEEE transactions on Computers","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ballé et al., 2016] Johannes Ballé, Valero Laparra, and Eero P Simoncelli. End-to-end optimized image compres- sion. arXiv preprint arXiv:1611.01704, 2016.","title":"End-to-end optimized image compression","authors":"Johannes Ballé, Valero Laparra, Eero P Simoncelli","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ballé et al., 2018] Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with a scale hyperprior. arXiv preprint arXiv:1802.01436, 2018.","title":"Variational image compression with a scale hyperprior","authors":"Johannes Ballé, David Minnen, Saurabh Singh, Sung, Jin Hwang, Nick Johnston","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ballé et al., 2020] Johannes Ballé, Philip A Chou, David Minnen, Saurabh Singh, Nick Johnston, Eirikur Agusts- son, Sung Jin Hwang, and George Toderici. Nonlinear transform coding. IEEE Journal of Selected Topics in Sig- nal Processing, 15(2):339-353, 2020.","title":"Nonlinear transform coding","authors":"Johannes Ballé, Philip A Chou, David Minnen, Saurabh Singh, Nick Johnston, Eirikur Agustsson, Sung, Jin Hwang, George Toderici","year":"2020","venue":"IEEE Journal of Selected Topics in Signal Processing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bégaint et al., 2020] Jean Bégaint, Fabien Racapé, Simon Feltman, and Akshay Pushparaja. Compressai: a pytorch library and evaluation platform for end-to-end compres- sion research. arXiv preprint arXiv:2011.03029, 2020.","title":"Compressai: a pytorch library and evaluation platform for end-to-end compression research","authors":"Bégaint","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bross et al., 2021] Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J Sullivan, and Jens-Rainer Ohm. Overview of the versatile video coding (vvc) stan- dard and its applications. IEEE Transactions on Circuits and Systems for Video Technology, 31(10):3736-3764, 2021.","title":"Overview of the versatile video coding (vvc) standard and its applications","authors":"Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J Sullivan, Jens-Rainer Ohm","year":"2021","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cheng et al., 2020] Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and atten- tion modules. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 7939-7948, 2020.","title":"Learned image compression with discretized gaussian mixture likelihoods and attention modules","authors":"Cheng","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2021] Zongyu Guo, Zhizheng Zhang, Runsen Feng, and Zhibo Chen. Causal contextual prediction for learned image compression. IEEE Transactions on Cir- cuits and Systems for Video Technology, 32(4):2329-2341, 2021.","title":"Causal contextual prediction for learned image compression","authors":"Guo","year":"2021","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2021] Dailan He, Yaoyan Zheng, Baocheng Sun, Yan Wang, and Hongwei Qin. Checkerboard context model for efficient learned image compression. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14771-14780, 2021.","title":"Checkerboard context model for efficient learned image compression","authors":"He","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2022] Huaibo Huang, Xiaoqiang Zhou, Jie Cao, Ran He, and Tieniu Tan. Vision transformer with super token sampling. arXiv:2211.11167, 2022.","title":"Vision transformer with super token sampling","authors":"Huaibo Huang, Xiaoqiang Zhou, Jie Cao, Ran He, Tieniu Tan","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jampani et al., 2018] Varun Jampani, Deqing Sun, Ming-Yu Liu, Ming-Hsuan Yang, and Jan Kautz. Superpixel sam- pling networks. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pages 352-368, 2018.","title":"Superpixel sampling networks","authors":"Varun Jampani, Deqing Sun, Ming-Yu Liu, Ming-Hsuan Yang, Jan Kautz","year":"2018","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2015] Shan Liu, Xiaozhong Xu, Shawmin Lei, and Kevin Jou. Overview of hevc extensions on screen content coding. APSIPA Transactions on Signal and In- formation Processing, 4:e10, 2015.","title":"Overview of hevc extensions on screen content coding","authors":"Shan Liu, Xiaozhong Xu, Shawmin Lei, Kevin Jou","year":"2015","venue":"APSIPA Transactions on Signal and Information Processing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2021] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF in- ternational conference on computer vision, pages 10012- 10022, 2021.","title":"Swin transformer: Hierarchical vision transformer using shifted windows","authors":"Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo","year":"2021","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2023] Jinming Liu, Heming Sun, and Jiro Katto. Learned image compression with mixed transformer-cnn architectures. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 1-10, 2023.","title":"Learned image compression with mixed transformer-cnn architectures","authors":"Jinming Liu, Heming Sun, Jiro Katto","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Min et al., 2021] Xiongkuo Min, Ke Gu, Guangtao Zhai, Xiaokang Yang, Wenjun Zhang, Patrick Le Callet, and Chang Wen Chen. Screen content quality assessment: overview, benchmark, and beyond. ACM Computing Sur- veys (CSUR), 54(9):1-36, 2021.","title":"Screen content quality assessment: overview, benchmark, and beyond","authors":"Min","year":"2021","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Minnen and Singh, 2020] David Minnen and Saurabh Singh. Channel-wise autoregressive entropy models for learned image compression. In 2020 IEEE Interna- tional Conference on Image Processing (ICIP), pages 3339-3343. IEEE, 2020.","title":"Channel-wise autoregressive entropy models for learned image compression","authors":"Singh Minnen, David Minnen, Saurabh Singh","year":"2020","venue":"IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Minnen et al., 2018] David Minnen, Johannes Ballé, and George D Toderici. Joint autoregressive and hierarchical priors for learned image compression. Advances in neural information processing systems, 31, 2018.","title":"Joint autoregressive and hierarchical priors for learned image compression","authors":"David Minnen, Johannes Ballé, George D Toderici","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mitrica et al., 2019] Iulia Mitrica, Eric Mercier, Christophe Ruellan, Attilio Fiandrotti, Marco Cagnazzo, and Béatrice Pesquet-Popescu. Very low bitrate semantic compression of airplane cockpit screen content. IEEE Transactions on Multimedia, 21(9):2157-2170, 2019.","title":"Very low bitrate semantic compression of airplane cockpit screen content","authors":"Iulia Mitrica, Eric Mercier, Christophe Ruellan, Attilio Fiandrotti, Marco Cagnazzo, Béatrice Pesquet-Popescu","year":"2019","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rabbani and Joshi, 2002] Majid Rabbani and Rajan Joshi. An overview of the jpeg 2000 still image compression standard. Signal processing: Image communication, 17(1):3-48, 2002.","title":"An overview of the jpeg 2000 still image compression standard","authors":"Majid Rabbani, Rajan Joshi","year":"2002","venue":"Signal processing: Image communication","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shen et al., 2022] Sheng Shen, Huanjing Yue, Jingyu Yang, and Kun Li. Itsrn++: Stronger and better implicit transformer network for continuous screen content image super-resolution. arXiv preprint arXiv:2210.08812, 2022.","title":"Itsrn++: Stronger and better implicit transformer network for continuous screen content image super-resolution","authors":"Sheng Shen, Huanjing Yue, Jingyu Yang, Kun Li","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sullivan et al., 2012] Gary J Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand. Overview of the high efficiency video coding (hevc) standard. IEEE Trans- actions on circuits and systems for video technology, 22(12):1649-1668, 2012.","title":"Overview of the high efficiency video coding (hevc) standard","authors":"Jens-Rainer Gary J Sullivan, Woo-Jin Ohm, Thomas Han, Wiegand","year":"2012","venue":"IEEE Transactions on circuits and systems for video technology","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tang et al., 2022] Tong Tang, Ling Li, Xiaoyu Wu, Ruizhi Chen, Haochen Li, Guo Lu, and Limin Cheng. Tsa-scc: text semantic-aware screen content coding with ultra low bitrate. IEEE Transactions on Image Processing, 31:2463- 2477, 2022.","title":"Tsa-scc: text semantic-aware screen content coding with ultra low bitrate","authors":"Tong Tang, Ling Li, Xiaoyu Wu, Ruizhi Chen, Haochen Li, Guo Lu, Limin Cheng","year":"2022","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wallace, 1992] Gregory K Wallace. The jpeg still picture compression standard. IEEE transactions on consumer electronics, 38(1):xviii-xxxiv, 1992.","title":"The jpeg still picture compression standard","authors":"Wallace Gregory","year":"1992","venue":"IEEE transactions on consumer electronics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2022] Meng Wang, Kai Zhang, Li Zhang, Yao- jun Wu, Yue Li, Junru Li, and Shiqi Wang. Transform skip inspired end-to-end compression for screen content image. In 2022 IEEE International Conference on Image Process- ing (ICIP), pages 3848-3852. IEEE, 2022.","title":"Transform skip inspired end-to-end compression for screen content image","authors":"Meng Wang, Kai Zhang, Li Zhang, Yaojun Wu, Yue Li, Junru Li, Shiqi Wang","year":"2022","venue":"IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xie et al., 2021] Yueqi Xie, Ka Leong Cheng, and Qifeng Chen. Enhanced invertible encoding for learned image compression. In Proceedings of the 29th ACM interna- tional conference on multimedia, pages 162-170, 2021.","title":"Enhanced invertible encoding for learned image compression","authors":"Xie","year":"2021","venue":"Proceedings of the 29th ACM international conference on multimedia","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xu et al., 2016] Xiaozhong Xu, Shan Liu, Tzu-Der Chuang, Yu-Wen Huang, Shaw-Min Lei, Krishnakanth Rapaka, Chao Pang, Vadim Seregin, Ye-Kui Wang, and Marta Kar- czewicz. Intra block copy in hevc screen content coding extensions. IEEE Journal on Emerging and Selected Top- ics in Circuits and Systems, 6(4):409-419, 2016.","title":"Intra block copy in hevc screen content coding extensions","authors":"Xiaozhong Xu, Shan Liu, Tzu-Der Chuang, Yu-Wen Huang, Shaw-Min Lei, Krishnakanth Rapaka, Chao Pang, Ye-Kui Vadim Seregin, Marta Wang, Karczewicz","year":"2016","venue":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zamanshoar Heris and Bajić, 2023] Rashid Zaman- shoar Heris and Ivan V Bajić. Multi-task learning for screen content image coding. arXiv e-prints, pages arXiv-2302, 2023.","title":"Multi-task learning for screen content image coding","authors":"Zamanshoar Rashid, Ivan V Heris, Bajić","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhu et al., 2022] Yinhao Zhu, Yang Yang, and Taco Co- hen. Transformer-based transform coding. In Interna- tional Conference on Learning Representations, 2022.","title":"Transformer-based transform coding","authors":"Zhu","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zou et al., 2022] Renjie Zou, Chunfeng Song, and Zhaox- iang Zhang. The devil is in the details: Window-based attention for image compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17492-17501, 2022.","title":"The devil is in the details: Window-based attention for image compression","authors":"Renjie Zou, Chunfeng Song, Zhaoxiang Zhang","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}