{"type":"ScholarlyArticle","title":"Domain Adversarial Learning for Color Constancy","authors":[{"name":"Zhifeng Zhang","slug":"zhifeng-zhang","given":"Zhifeng","family":"Zhang","affiliation":"School of Computer Science (National Pilot Software Engineering School) , Beijing University of Posts and Telecommunications"},{"name":"Xuejing Kang","slug":"xuejing-kang","given":"Xuejing","family":"Kang","affiliation":"School of Computer Science (National Pilot Software Engineering School) , Beijing University of Posts and Telecommunications"},{"name":"Anlong Ming","slug":"anlong-ming","given":"Anlong","family":"Ming","affiliation":"School of Computer Science (National Pilot Software Engineering School) , Beijing University of Posts and Telecommunications"}],"abstract":"Color Constancy aims to eliminate the color cast of RAW images caused by non-neutral illuminants. Though contemporary approaches based on convolutional neural networks significantly improve illuminant estimation, they suffer from the seriously insufficient data problem. To solve this problem by effectively utilizing multi-domain data, we propose the Domain Adversarial Learning  Color Constancy (DALCC) which consists of the Domain Adversarial Learning Branch (DALB) and the Feature Reweighting Module (FRM). In DALB, the Camera Domain Classifier and the feature extractor compete against each other in an adversarial way to encourage the emergence of domain-invariant features. At the same time, the Illuminant Transformation Module performs color space conversion to solve the inconsistent color space problem caused by those domain-invariant features. They collaboratively avoid model degradation of multi-device training caused by the domain discrepancy of feature distribution, which enables our DALCC to benefit from multi-domain data. Besides, to better utilize multi-domain data, we propose the FRM that reweights the feature map to suppress Non-Primary Illuminant regions, which reduces the influence of misleading illuminant information. Experiments show that the proposed DALCC can more effectively take advantage of multi-domain data and thus achieve state-of-the-art performance on commonly used benchmark datasets.","doi":"10.24963/ijcai.2022/236","url":"https://ijcai.org/proceedings/2022/236","pdf":"https://ijcai.org/proceedings/2022/0236.pdf","markdown":"https://ijcai.org/proceedings/2022/236.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/236","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":236,"pages":{"first":1693,"last":1699},"keywords":["Computer Vision: Machine Learning for Vision","Machine Learning: Convolutional Networks","Computer Vision: Computational photography","Machine Learning: Adversarial Machine Learning","Machine Learning: Multi-task and Transfer Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986746","length_sec":73,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984889","length_sec":294,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Afifi and Brown, 2019a] M. Afifi and M. S. Brown. Sensor- independent illumination estimation for dnn models. In BMVC 2019, 2019.","title":"Sensorindependent illumination estimation for dnn models","authors":"Brown Afifi, M Afifi, M S Brown","year":"2019","venue":"BMVC 2019","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Afifi and Brown, 2019b] Mahmoud Afifi and Michael S Brown. What else can fool deep learning? address- ing color constancy errors on deep neural network per- formance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 243-252, 2019.","title":"What else can fool deep learning? addressing color constancy errors on deep neural network performance","authors":"Brown Afifi, Mahmoud Afifi, Michael S Brown","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Afifi et al., 2019] Mahmoud Afifi, Abhijith Punnappurath, Graham Finlayson, and Michael S Brown. As-projective- as-possible bias correction for illumination estimation al- gorithms. JOSA A, 36(1):71-78, 2019.","title":"As-projectiveas-possible bias correction for illumination estimation algorithms","authors":"Mahmoud Afifi, Abhijith Punnappurath, Graham Finlayson, Michael S Brown","year":"2019","venue":"JOSA A","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Afifi et al., 2021] Mahmoud Afifi, Jonathan T Barron, Chloe LeGendre, Yun-Ta Tsai, and Francois Bleibel. Cross-camera convolutional color constancy. In Proc. IEEE Int. Conf. Comput. Vis., pages 1981-1990, 2021.","title":"Cross-camera convolutional color constancy","authors":"Mahmoud Afifi, Jonathan T Barron, Chloe Legendre, Yun-Ta Tsai, Francois Bleibel","year":"2021","venue":"Proc. IEEE Int. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Banic and Loncaric, 2015] Nikola Banic and Sven Lon- caric. Color dog-guiding the global illumination estima- tion to better accuracy. In VISAPP, pages 129-135, 2015.","title":"Color dog-guiding the global illumination estimation to better accuracy","authors":"Loncaric Banic","year":"2015","venue":"VISAPP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Banić et al., 2017] Nikola Banić, Karlo Koščević, and Sven Lončarić. Unsupervised learning for color constancy. arXiv preprint arXiv:1712.00436, 2017.","title":"Unsupervised learning for color constancy","authors":"Nikola Banić, Karlo Koščević, Sven Lončarić","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Barron and Tsai, 2017] Jonathan T Barron and Yun-Ta Tsai. Fast fourier color constancy. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 886-894, 2017.","title":"Fast fourier color constancy","authors":"Jonathan T Barron, Yun-Ta Tsai","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ben-David et al., 2010] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79(1):151-175, 2010.","title":"A theory of learning from different domains","authors":"Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, Jennifer Wortman Vaughan","year":"2010","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Bianco and Cusano, 2019] Simone Bianco and Claudio Cu- sano. Quasi-unsupervised color constancy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 12212-12221, 2019.","title":"Quasi-unsupervised color constancy","authors":"Simone Bianco, Claudio Cusano","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Buchsbaum, 1980] Gershon Buchsbaum. A spatial proces- sor model for object colour perception. Journal of the Franklin institute, 310(1):1-26, 1980.","title":"A spatial processor model for object colour perception","authors":"Gershon Buchsbaum","year":"1980","venue":"Journal of the Franklin institute","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Chen et al., 2015] Yu-Hsiu Chen, Ting-Hsuan Chao, Sheng-Yi Bai, Yen-Liang Lin, Wen-Chin Chen, and Winston H Hsu. Filter-invariant image classification on social media photos. In Proc. ACM Conf. Multimed., pages 855-858, 2015.","title":"Filter-invariant image classification on social media photos","authors":"Yu-Hsiu Chen, Ting-Hsuan Chao, Sheng-Yi Bai, Yen-Liang Lin, Wen-Chin Chen, Winston H Hsu","year":"2015","venue":"Proc. ACM Conf. Multimed","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Cheng et al., 2014] Dongliang Cheng, Dilip K Prasad, and Michael S Brown. Illuminant estimation for color con- stancy: why spatial-domain methods work and the role of the color distribution. JOSA A, 31(5):1049-1058, 2014.","title":"Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution","authors":"Dongliang Cheng, Dilip K Prasad, Michael S Brown","year":"2014","venue":"JOSA A","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Finlayson and Trezzi, 2004] Graham D Finlayson and Elis- abetta Trezzi. Shades of gray and colour constancy. In Color and Imaging Conference, volume 2004, pages 37- 41. Society for Imaging Science and Technology, 2004.","title":"Shades of gray and colour constancy","authors":"D Graham, Elisabetta Finlayson, Trezzi","year":"2004","venue":"Color and Imaging Conference","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Geusebroek et al., 2003] Jan-Mark Geusebroek, Rein van den Boomgaard, Arnold WM Smeulders, and Theo Gevers. Color constancy from physical principles. Pattern Recognition Letters, 24(11):1653-1662, 2003.","title":"Color constancy from physical principles","authors":"Geusebroek","year":"2003","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gijsenij and Gevers, 2010] Arjan Gijsenij and Theo Gevers. Color constancy using natural image statistics and scene semantics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4):687-698, 2010.","title":"Color constancy using natural image statistics and scene semantics","authors":"Gevers Gijsenij, Arjan Gijsenij, Theo Gevers","year":"2010","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hernandez et al., 2020] Daniel Hernandez, Sarah Parisot, Benjamin Busam, Ales Leonardis, Gregory Slabaugh, and Steven McDonagh. A multi-hypothesis approach to color constancy. In Proceedings of the Proc. IEEE Conf. Com- put. Vis. Pattern Recog., pages 2270-2280, 2020.","title":"A multi-hypothesis approach to color constancy","authors":"Daniel Hernandez, Sarah Parisot, Benjamin Busam, Ales Leonardis, Gregory Slabaugh, Steven Mcdonagh","year":"2020","venue":"Proceedings of the Proc. IEEE Conf. Comput. Vis. Pattern Recog","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hu et al., 2017] Yuanming Hu, Baoyuan Wang, and Stephen Lin. Fc4: Fully convolutional color constancy with confidence-weighted pooling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4085-4094, 2017.","title":"Fc4: Fully convolutional color constancy with confidence-weighted pooling","authors":"Yuanming Hu, Baoyuan Wang, Stephen Lin","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"Edwin H Land. The retinex theory of color vi- sion. Scientific american, 237(6):108-129, 1977.","title":"The retinex theory of color vision","authors":"H Edwin, Land","year":"1977","venue":"Scientific american","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lo et al., 2021] Yi-Chen Lo, Chia-Che Chang, Hsuan-Chao Chiu, Yu-Hao Huang, Chia-Ping Chen, Yu-Lin Chang, and Kevin Jou. Clcc: Contrastive learning for color constancy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8053-8063, 2021.","title":"Clcc: Contrastive learning for color constancy","authors":"Yi-Chen Lo, Chia-Che Chang, Hsuan-Chao Chiu, Yu-Hao Huang, Chia-Ping Chen, Yu-Lin Chang, Kevin Jou","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lou et al., 2015] Zhongyu Lou, Theo Gevers, Ninghang Hu, Marcel P Lucassen, et al. Color constancy by deep learning. In BMVC, pages 76-1, 2015.","title":"Color constancy by deep learning","authors":"Lou","year":"2015","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tang et al., 2022] Yuxiang Tang, Xuejing Kang, Chunxiao Li, Zhaowen Lin, and Anlong Ming. Transfer learning for color constancy via statistic perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, 2022.","title":"Transfer learning for color constancy via statistic perspective","authors":"Yuxiang Tang, Xuejing Kang, Chunxiao Li, Zhaowen Lin, Anlong Ming","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Van De Weijer et al., 2007] Joost Van De Weijer, Theo Gevers, and Arjan Gijsenij. Edge-based color constancy. IEEE Trans. Image Process., 16(9):2207-2214, 2007.","title":"Edge-based color constancy","authors":"Van De Weijer","year":"2007","venue":"IEEE Trans. Image Process","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xiao et al., 2020] Jin Xiao, Shuhang Gu, and Lei Zhang. Multi-domain learning for accurate and few-shot color constancy. In Proc. IEEE Conf. Comput. Vis. Pattern Recog., pages 3258-3267, 2020.","title":"Multi-domain learning for accurate and few-shot color constancy","authors":"Jin Xiao, Shuhang Gu, Lei Zhang","year":"2020","venue":"Proc. IEEE Conf. Comput. Vis. Pattern Recog","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xu et al., 2020] Bolei Xu, Jingxin Liu, Xianxu Hou, Bozhi Liu, and Guoping Qiu. End-to-end illuminant estimation based on deep metric learning. In Proc. IEEE Conf. Com- put. Vis. Pattern Recog., pages 3616-3625, 2020.","title":"End-to-end illuminant estimation based on deep metric learning","authors":"Xu","year":"2020","venue":"Proc. IEEE Conf. Comput. Vis. Pattern Recog","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yu et al., 2020] Huanglin Yu, Ke Chen, Kaiqi Wang, Yanlin Qian, Zhaoxiang Zhang, and Kui Jia. Cascading convolu- tional color constancy. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 34, pages 12725- 12732, 2020.","title":"Cascading convolutional color constancy","authors":"Huanglin Yu, Ke Chen, Kaiqi Wang, Yanlin Qian, Zhaoxiang Zhang, Kui Jia","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}