{"type":"ScholarlyArticle","title":"Customized Positional Encoding to Combine Static and Time-varying Data in Robust Representation Learning for Crop Yield Prediction","authors":[{"name":"Qinqing Liu","slug":"qinqing-liu","given":"Qinqing","family":"Liu","affiliation":"Computer Science and Engineering Department , University of Connecticut"},{"name":"Fei Dou","slug":"fei-dou","given":"Fei","family":"Dou","affiliation":"Computer Science and Engineering Department , University of Connecticut"},{"name":"Meijian Yang","slug":"meijian-yang","given":"Meijian","family":"Yang","affiliation":"Center for Climate Systems Research , Climate School , Earth Institute , Columbia University"},{"name":"Ezana Amdework","slug":"ezana-amdework","given":"Ezana","family":"Amdework","affiliation":"Department of Sociology , Addis Ababa University"},{"name":"Guiling Wang","slug":"guiling-wang","given":"Guiling","family":"Wang","affiliation":"Civil and Environmental Engineering Department , University of Connecticut"},{"name":"Jinbo Bi","slug":"jinbo-bi","given":"Jinbo","family":"Bi","affiliation":"Computer Science and Engineering Department , University of Connecticut"}],"abstract":"Accurate prediction of crop yield under the conditions of climate change is crucial to ensure food security. Transformers have shown remarkable success in modeling sequential data and hold the potential for improving crop yield prediction. To understand how weather and meteorological sequence variables affect crop yield, the positional encoding used in Transformers is typically shared across different sample sequences. We argue that it is necessary and beneficial to differentiate the positional encoding for distinct samples based on time-invariant properties of the sequences. Particularly, the sequence variables influencing crop yield vary according to static variables such as geographical locations. Sample data from southern areas may benefit from more tailored positional encoding different from that for northern areas. We propose a novel transformer based architecture for accurate and robust crop yield prediction, by introducing a Customized Positional Encoding (CPE) that encodes a sequence adaptively according to static information associated with the sequence. Empirical studies demonstrate the effectiveness of the proposed novel architecture and show that partially lin-\r\nearized attention better captures the bias introduced by side information than softmax re-weighting. The resultant crop yield prediction model is robust to climate change, with mean-absolute-error reduced by up to 26% compared to the best baseline model in extreme drought years.","doi":"10.24963/ijcai.2023/676","url":"https://ijcai.org/proceedings/2023/676","pdf":"https://ijcai.org/proceedings/2023/0676.pdf","markdown":"https://ijcai.org/proceedings/2023/676.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/676","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI for Good","track":"AI for Good","number":676,"pages":{"first":6094,"last":6102},"keywords":["AI for Good: Multidisciplinary Topics and Applications","AI for Good: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Abatzoglou, 2013] John T Abatzoglou. Development of gridded surface meteorological data for ecological appli- cations and modelling. International Journal of Climatol- ogy, 33(1):121-131, 2013.","title":"Development of gridded surface meteorological data for ecological applications and modelling","authors":"John T Abatzoglou","year":"2013","venue":"International Journal of Climatology","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Attia et al., 2021] Ahmed Attia, Salah El-Hendawy, Nasser Al-Suhaibani, Muhammad Usman Tahir, Muhammad Mubushar, Murilo dos Santos Vianna, Hayat Ullah, El- sayed Mansour, and Avishek Datta. Sensitivity of the dssat model in simulating maize yield and soil carbon dynam- ics in arid mediterranean climate: Effect of soil, genotype and crop management. Field crops research, 260:107981, 2021.","title":"Sensitivity of the dssat model in simulating maize yield and soil carbon dynamics in arid mediterranean climate: Effect of soil, genotype and crop management","authors":"Ahmed Attia, Salah El-Hendawy, Nasser Al-Suhaibani, Muhammad Usman Tahir, Muhammad Mubushar, Murilo, Santos Vianna, Hayat Ullah, Elsayed Mansour, Avishek Datta","year":"2021","venue":"Field crops research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Boyer et al., 2013] JS Boyer, P Byrne, KG Cassman, M Cooper, D Delmer, T Greene, F Gruis, J Habben, N Hausmann, N Kenny, et al. The us drought of 2012 in perspective: a call to action. Global Food Security, 2(3):139-143, 2013.","title":"The us drought of 2012 in perspective: a call to action","authors":"Boyer","year":"2013","venue":"Global Food Security","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cao et al., 2018] Peiyu Cao, Chaoqun Lu, and Zhen Yu. Historical nitrogen fertilizer use in agricultural ecosystems of the contiguous united states during 1850-2015: applica- tion rate, timing, and fertilizer types. Earth System Science Data, 10(2):969-984, 2018.","title":"Historical nitrogen fertilizer use in agricultural ecosystems of the contiguous united states during 1850-2015: application rate, timing, and fertilizer types","authors":"Peiyu Cao, Chaoqun Lu, Zhen Yu","year":"2018","venue":"Earth System Science Data","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chu et al., 2021] Xiangxiang Chu, Zhi Tian, Bo Zhang, Xinlong Wang, Xiaolin Wei, Huaxia Xia, and Chunhua Shen. Conditional positional encodings for vision trans- formers. arXiv preprint arXiv:2102.10882, 2021.","title":"Conditional positional encodings for vision transformers","authors":"Xiangxiang Chu, Zhi Tian, Bo Zhang, Xinlong Wang, Xiaolin Wei, Huaxia Xia, Chunhua Shen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[CSA, 2021] CSA. Agricultural Sample Survey 2020/21 (2013","title":"CSA. Agricultural Sample Survey","authors":"Csa","year":"2013","venue":"CSA. Agricultural Sample Survey","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"E.C). Report on Area and Production of Major Crops, volume I. Statistical Bulletin 590, Central Statis- tical Authority, Addis Ababa, 2021.","title":"Report on Area and Production of Major Crops, volume I","authors":"E C","year":"2021","venue":"Statistical Bulletin","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dwivedi and Bresson, 2020] Vijay Prakash Dwivedi and Xavier Bresson. A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699, 2020.","title":"A generalization of transformer networks to graphs","authors":"Vijay Prakash, Dwivedi, Xavier Bresson","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[FSIN and GRFC, 2023] FSIN and GRFC. Global report on food crises 2023. https://www.fsinplatform.org/sites/ default/files/resources/files/GRFC2023-hi-res.pdf, 2023. Accessed: 2023-06-05.","title":"FSIN and GRFC. Global report on food crises 2023","authors":"Grfc Fsin","year":"2023","venue":null,"doi":null,"url":"https://www.fsinplatform.org/sites/default/files/resources/files/GRFC2023-hi-res.pdf","internal_url":null},{"position":10,"raw":"[He et al., 2021] Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. Transreid: Transformer- based object re-identification. In Proceedings of the IEEE/CVF international conference on computer vision, pages 15013-15022, 2021.","title":"Transreid: Transformerbased object re-identification","authors":"Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, Wei Jiang","year":"2021","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hersbach et al., 2018] Hans Hersbach, Bill Bell, Paul Berrisford, Gionata Biavati, András Horányi, Joaquín Muñoz Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Iryna Rozum, et al. Era5 hourly data on single levels from 1979 to present. Copernicus climate change service (c3s) climate data store (cds), 10(10.24381), 2018.","title":"Era5 hourly data on single levels from 1979 to present","authors":"Hersbach","year":"2018","venue":"Copernicus climate change service (c3s) climate data store","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2020] Zhiheng Huang, Davis Liang, Peng Xu, and Bing Xiang. Improve transformer models with better relative position embeddings. arXiv preprint arXiv:2009.13658, 2020.","title":"Improve transformer models with better relative position embeddings","authors":"Zhiheng Huang, Davis Liang, Peng Xu, Bing Xiang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jiang et al., 2020] Hao Jiang, Hao Hu, Renhai Zhong, Jin- fan Xu, Jialu Xu, Jingfeng Huang, Shaowen Wang, Yibin Ying, and Tao Lin. A deep learning approach to conflating heterogeneous geospatial data for corn yield estimation: A case study of the us corn belt at the county level. Global change biology, 26(3):1754-1766, 2020.","title":"A deep learning approach to conflating heterogeneous geospatial data for corn yield estimation: A case study of the us corn belt at the county level","authors":"Hao Jiang, Hao Hu, Renhai Zhong, Jinfan Xu, Jialu Xu, Jingfeng Huang, Shaowen Wang, Yibin Ying, Tao Lin","year":"2020","venue":"Global change biology","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jin et al., 2019] Cui Jin, Xue Luo, Xiangming Xiao, Jinwei Dong, Xueming Li, Jun Yang, and Deyu Zhao. The 2012 flash drought threatened us midwest agroecosystems. Chi- nese Geographical Science, 29:768-783, 2019.","title":"The 2012 flash drought threatened us midwest agroecosystems","authors":"Cui Jin, Xue Luo, Xiangming Xiao, Jinwei Dong, Xueming Li, Jun Yang, Deyu Zhao","year":"2019","venue":"Chinese Geographical Science","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jones et al., 1986] C Allan Jones, James Robert Kiniry, and PT Dyke. CERES-Maize: A simulation model of maize growth and development. Texas A& M University Press, 1986.","title":"CERES-Maize: A simulation model of maize growth and development","authors":"Allan Jones, James Robert Kiniry, Dyke","year":"1986","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Katharopoulos et al., 2020] Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and Franc ¸ois Fleuret. Transformers are rnns: Fast autoregressive transformers with linear attention. In International Conference on Machine Learning, pages 5156-5165. PMLR, 2020.","title":"Transformers are rnns: Fast autoregressive transformers with linear attention","authors":"Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, Franc ¸ois, Fleuret","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ke et al., 2020] Guolin Ke, Di He, and Tie-Yan Liu. Re- thinking positional encoding in language pre-training. arXiv preprint arXiv:2006.15595, 2020.","title":"Rethinking positional encoding in language pre-training","authors":"Guolin Ke, Di He, Tie-Yan Liu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Khaki et al., 2020] Saeed Khaki, Lizhi Wang, and Sotirios V Archontoulis. A cnn-rnn framework for crop yield prediction. Frontiers in Plant Science, 10:1750, 2020.","title":"A cnn-rnn framework for crop yield prediction","authors":"Saeed Khaki, Lizhi Wang, Sotirios V Archontoulis","year":"2020","venue":"Frontiers in Plant Science","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kreuzer et al., 2021] Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, and Prudencio Tossou. Rethinking graph transformers with spectral atten- tion. Advances in Neural Information Processing Systems, 34:21618-21629, 2021.","title":"Rethinking graph transformers with spectral attention","authors":"Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, Prudencio Tossou","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2021] Yang Li, Si Si, Gang Li, Cho-Jui Hsieh, and Samy Bengio. Learnable fourier features for multi- dimensional spatial positional encoding. Advances in Neural Information Processing Systems, 34:15816-15829, 2021.","title":"Learnable fourier features for multidimensional spatial positional encoding","authors":"Yang Li, Si Si, Gang Li, Cho-Jui Hsieh, Samy Bengio","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2021] Chang Liu, Xiaoguang Li, Guohao Cai, Zhenhua Dong, Hong Zhu, and Lifeng Shang. Noninva- sive self-attention for side information fusion in sequential recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 4249-4256, 2021.","title":"Noninvasive self-attention for side information fusion in sequential recommendation","authors":"Chang Liu, Xiaoguang Li, Guohao Cai, Zhenhua Dong, Hong Zhu, Lifeng Shang","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2022a] Qinqing Liu, Meijian Yang, Koushan Mo- hammadi, Dongjin Song, Jinbo Bi, and Guiling Wang. Machine learning crop yield models based on meteorolog- ical features and comparison with a process-based model. Artificial Intelligence for the Earth Systems, 1(4):e220002, 2022.","title":"Machine learning crop yield models based on meteorological features and comparison with a process-based model","authors":"Liu","year":"2022","venue":"Artificial Intelligence for the Earth Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2022b] Yuanyuan Liu, Shaoqiang Wang, Jinghua Chen, Bin Chen, Xiaobo Wang, Dongze Hao, and Leigang Sun. Rice yield prediction and model interpretation based on satellite and climatic indicators using a transformer method. Remote Sensing, 14(19):5045, 2022.","title":"Rice yield prediction and model interpretation based on satellite and climatic indicators using a transformer method","authors":"Liu","year":"2022","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Luo et al., 2022] Shengjie Luo, Shanda Li, Shuxin Zheng, Tie-Yan Liu, Liwei Wang, and Di He. Your transformer may not be as powerful as you expect. arXiv preprint arXiv:2205.13401, 2022.","title":"Your transformer may not be as powerful as you expect","authors":"Luo","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Qi et al., 2021] Xinyuan Qi, Kai Hou, Tong Liu, Zhongzhong Yu, Sihao Hu, and Wenwu Ou. From known to unknown: Knowledge-guided transformer for time-series sales forecasting in alibaba. arXiv preprint arXiv:2109.08381, 2021.","title":"From known to unknown: Knowledge-guided transformer for time-series sales forecasting in alibaba","authors":"Xinyuan Qi, Kai Hou, Tong Liu, Zhongzhong Yu, Sihao Hu, Wenwu Ou","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Qin et al., 2022] Zhen Qin, Weixuan Sun, Hui Deng, Dongxu Li, Yunshen Wei, Baohong Lv, Junjie Yan, Ling- peng Kong, and Yiran Zhong. cosformer: Rethinking soft- max in attention. arXiv preprint arXiv:2202.08791, 2022.","title":"cosformer: Rethinking softmax in attention","authors":"Zhen Qin, Weixuan Sun, Hui Deng, Dongxu Li, Yunshen Wei, Baohong Lv, Junjie Yan, Lingpeng Kong, Yiran Zhong","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Raffel et al., 2020] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1-67, 2020.","title":"Exploring the limits of transfer learning with a unified text-to-text transformer","authors":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu","year":"2020","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Raisi et al., 2021] Zobeir Raisi, Mohamed A Naiel, Georges Younes, Steven Wardell, and John Zelek. 2lspe: 2d learnable sinusoidal positional encoding using transformer for scene text recognition. In 2021 18th Conference on Robots and Vision (CRV), pages 119-126. IEEE, 2021.","title":"2lspe: 2d learnable sinusoidal positional encoding using transformer for scene text recognition","authors":"Raisi","year":"2021","venue":"18th Conference on Robots and Vision (CRV)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Schnitkey, 2013] Gary Schnitkey. Corn-to-soybean yield ra- tios: History and the future. farmdoc daily, 3(181), 2013.","title":"Corn-to-soybean yield ratios: History and the future","authors":"Gary Schnitkey","year":"2013","venue":"farmdoc daily","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Shang et al., 2021] Chao Shang, Qinqing Liu, Qianqian Tong, Jiangwen Sun, Minghu Song, and Jinbo Bi. Multi- view spectral graph convolution with consistent edge at- tention for molecular modeling. Neurocomputing, 445:12- 25, 2021.","title":"Multiview spectral graph convolution with consistent edge attention for molecular modeling","authors":"Chao Shang, Qinqing Liu, Qianqian Tong, Jiangwen Sun, Minghu Song, Jinbo Bi","year":"2021","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Shaw et al., 2018] Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representa- tions. arXiv preprint arXiv:1803.02155, 2018.","title":"Self-attention with relative position representations","authors":"Peter Shaw, Jakob Uszkoreit, Ashish Vaswani","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sun et al., 2019] Jie Sun, Liping Di, Ziheng Sun, Yonglin Shen, and Zulong Lai. County-level soybean yield pre- diction using deep cnn-lstm model. Sensors, 19(20):4363, 2019.","title":"County-level soybean yield prediction using deep cnn-lstm model","authors":"Jie Sun, Liping Di, Ziheng Sun, Yonglin Shen, Zulong Lai","year":"2019","venue":"Sensors","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Veličković et al., 2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Xie et al., 2022] Yueqi Xie, Peilin Zhou, and Sunghun Kim. Decoupled side information fusion for sequential recom- mendation. arXiv preprint arXiv:2204.11046, 2022.","title":"Decoupled side information fusion for sequential recommendation","authors":"Xie","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Yang et al., 2020] Meijian Yang, Guiling Wang, Kazi Farzan Ahmed, Berihun Adugna, Michael Eggen, Ezana Atsbeha, Liangzhi You, Jawoo Koo, and Em- manouil Anagnostou. The role of climate in the trend and variability of ethiopia's cereal crop yields. Science of The Total Environment, 723:137893, 2020.","title":"The role of climate in the trend and variability of ethiopia's cereal crop yields","authors":"Meijian Yang, Guiling Wang, Farzan Kazi, Berihun Ahmed, Michael Adugna, Ezana Eggen, Liangzhi Atsbeha, Jawoo You, Emmanouil Koo, Anagnostou","year":"2020","venue":"Science of The Total Environment","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Yang et al., 2021] Meijian Yang, Guiling Wang, Rehenuma Lazin, Xinyi Shen, and Emmanouil Anagnostou. Impact of planting time soil moisture on cereal crop yield in the upper blue nile basin: A novel insight towards agricul- tural water management. Agricultural Water Management, 243:106430, 2021.","title":"Impact of planting time soil moisture on cereal crop yield in the upper blue nile basin: A novel insight towards agricultural water management","authors":"Yang","year":"2021","venue":"Agricultural Water Management","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Ying et al., 2021] Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, 34:28877-28888, 2021.","title":"Do transformers really perform badly for graph representation?","authors":"Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, Tie-Yan Liu","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhang et al., 2019] Liangliang Zhang, Zhao Zhang, Yuchuan Luo, Juan Cao, and Fulu Tao. Combining optical, fluorescence, thermal satellite, and environmental data to predict county-level maize yield in china using machine learning approaches. Remote Sensing, 12(1):21, 2019.","title":"Combining optical, fluorescence, thermal satellite, and environmental data to predict county-level maize yield in china using machine learning approaches","authors":"Liangliang Zhang, Zhao Zhang, Yuchuan Luo, Juan Cao, Fulu Tao","year":"2019","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhang et al., 2022] Haojie Zhang, Mingfei Liang, Ruobing Xie, Zhenlong Sun, Bo Zhang, and Leyu Lin. Improve transformer pre-training with decoupled directional rela- tive position encoding and representation differentiations. arXiv preprint arXiv:2210.04246, 2022.","title":"Improve transformer pre-training with decoupled directional relative position encoding and representation differentiations","authors":"Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zheng et al., 2022] Yanwei Zheng, Zengrui Zhao, Xiaowei Yu, and Dongxiao Yu. Template-aware transformer for person reidentification. Computational Intelligence and Neuroscience, 2022, 2022.","title":"Template-aware transformer for person reidentification","authors":"Yanwei Zheng, Zengrui Zhao, Xiaowei Yu, Dongxiao Yu","year":"2022","venue":"Computational Intelligence and Neuroscience","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}