--- title: "Progressive Domain-Independent Feature Decomposition Network for Zero-Shot Sketch-Based Image Retrieval" authors: - "Xinxun Xu" - "Muli Yang" - "Yanhua Yang" - "Hao Wang" doi: "10.24963/ijcai.2020/137" url: "https://ijcai.org/proceedings/2020/137" pdf: "https://ijcai.org/proceedings/2020/0137.pdf" venue: "Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence" year: 2020 pages: "984-990" track: "Main track" keywords: - "Machine Learning: Deep Generative Models" - "Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation" - "Machine Learning: Deep Learning" --- # Progressive Domain-Independent Feature Decomposition Network for Zero-Shot Sketch-Based Image Retrieval Xinxun Xu, Muli Yang, Yanhua Yang, Hao Wang *Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence*, pages 984-990. DOI: [10.24963/ijcai.2020/137](https://doi.org/10.24963/ijcai.2020/137) ## Abstract Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR) is a specific cross-modal retrieval task for searching natural images given free-hand sketches under the zero-shot scenario. Most existing methods solve this problem by simultaneously projecting visual features and semantic supervision into a low-dimensional common space for efficient retrieval. However, such low-dimensional projection destroys the completeness of semantic knowledge in original semantic space, so that it is unable to transfer useful knowledge well when learning semantic features from different modalities. Moreover, the domain information and semantic information are entangled in visual features, which is not conducive for cross-modal matching since it will hinder the reduction of domain gap between sketch and image. In this paper, we propose a Progressive Domain-independent Feature Decomposition (PDFD) network for ZS-SBIR. Specifically, with the supervision of original semantic knowledge, PDFD decomposes visual features into domain features and semantic ones, and then the semantic features are projected into common space as retrieval features for ZS-SBIR. The progressive projection strategy maintains strong semantic supervision. Besides, to guarantee the retrieval features to capture clean and complete semantic information, the cross-reconstruction loss is introduced to encourage that any combinations of retrieval features and domain features can reconstruct the visual features. Extensive experiments demonstrate the superiority of our PDFD over state-of-the-art competitors. ## Full text ## 1 Introduction With the explosive growth of image contents in our real world, image retrieval has been playing an important role in many fields, such as e-commerce, medical diagnosis and remote sensing. Conventional image retrieval requires providing textual descriptions, which are difficult to be obtained in many real-world cases. On mobile devices, image retrieval with free-hand sketches, for delivering targeted candidates visually and concisely, has attracted increasing attention and formed the area of Sketch-Based Image Retrieval (SBIR). ∗Contact Author Figure 1: The ways in Figure 1(a) and Figure 1(b) simultaneously project visual features and label/semantic supervision into a lowdimensional common space for efficient retrieval. The Figure 1(c) shows our way that first aligns sketch and image to semantic embedding explicitly and then projects them into common space. ![Image](https://ijcai.org/proceedings/2020/fulltext/137/image_000000_8092d1275642c02475df4da8ede3379baf44b3a2955a676362d2967e4bd47a86.png) Since it is difficult to guarantee that all categories can be trained in realistic scenarios, unsatisfactory performance is often yielded when testing on unseen categories. In view of this, a more realistic setting is emerged, namely ZS-SBIR, which combines Zero-Shot Learning (ZSL) and SBIR for real-world applications. The ZS-SBIR is a retrieval task under zero-shot scenario, and people can quickly draw sketches on the mobile for retrieval, so it attracted increasing attention in real world. However, ZS-SBIR is extremely challenging since it simultaneously needs to deal with cross-modal matching, significant domain gap, as well as limited knowledge of unseen classes. The traditional SBIR methods [Liu et al., 2017a; Zhang et al., 2018] cannot directly address these problems effectively since they over-fit the source domain and meanwhile neglect the unseen categories. On the contrary, traditional ZSL [Kodirov et al., 2017; Zhang and Saligrama, 2016] methods often focus on solving single-modal problems. Therefore, ZS-SBIR tries to solve these problems sufficiently by combining the advantages of ZSL and SBIR. As shown in Figure 1(a) and Figure 1(b), previous work attempted to overcome these challenges through simultaneously projecting sketch/image features and label/semantic supervision to a low-dimensional common space. However, these strategies deteriorate the original semantic knowledge, because the low-dimensional projection maps complete semantic embedding from original semantic space to semantically incomplete low-dimensional space. Hence, as shown in Figure 1(c), we present progressive projection strategy that first learns semantic features with original semantic supervision, and then projects them into a common retrieval space, which is beneficial to knowledge transfer due to the strong semantic supervision can be maintained. Another issue is that, the domain information and semantic information are entangled in visual features, and the distribution of two different domains are highly heterogeneous, which seriously hinders the reduction of domain gap between sketch and image, making cross-modal matching difficult. Since semantic information remains the same expression in different domains, we argue that only semantic information is crucial for cross-modal matching. To this end, we decompose the visual features to attain domain-independent retrieval features that only contain semantic information. Figure 2: Flowchart of our proposed PDFD. First, it decomposes the visual features into semantic features and domain features, where the semantic features are learned in an adversarial fashion, while the domain features are learned under the constraint of modality classifier. Subsequently, the semantic features are projected into a common space as retrieval features. Moreover, the semantic embedding derived from the text-based embedding and hierarchical embedding serve as true examples to the discriminator. Meanwhile, the cross-reconstruction loss guarantees that the retrieval features only contain high-level knowledge, which is beneficial to reducing the interference of domain features. ![Image](https://ijcai.org/proceedings/2020/fulltext/137/image_000001_716ec4fb794678798a7e465e1eac68b045b0fdd82547ab535abd1a860c0f6bc1.png) In this paper, we have proposed a Progressive DomainIndependent Feature Decomposition (PDFD) network for ZSSBIR task. First, PDFD decomposes visual features into semantic features and domain features, where the semantic features are adversarially learned with the supervision of original semantic embedding, while the domain features are learned with a modality classifier. Subsequently, the learned semantic features are projected into a common space as retrieval features under the category and modality supervision. Besides, in order to reduce the domain interference for cross-modal matching, we introduce cross-reconstruction loss to encourage the retrieval features capture clean and complete semantic information. It is expected that such retrieval features can reconstruct the sketch or image visual features by combining with sketch-domain features or image-domain features. In this network, the parameters of decoders and encoders for sketches and images are shared. The main contributions of this work are summarized: - We propose a feature decomposition model to effectively reduce the domain gap by generating domainindependent retrieval features with a novel crossreconstruction loss. - The proposed progressive projection strategy preserves the strong semantic supervision when generating retrieval features, which is beneficial to knowledge transfer under the zero-shot scenario. - Extensive experiments conducted on two popular largescale datasets demonstrate that our proposed PDFD significantly outperforms state-of-the-art methods. ## 2 RelatedWork In this section, we briefly review the prior literature in the fields of SBIR, ZSL and ZS-SBIR. ## 2.1 Sketch-Based Image Retrieval The existing SBIR approaches can be mainly divided into two categories: hand-crafted features based methods and deep learning based ones. The hand-crafted features based methods attempt to bridge the domain gap by using edge-maps extracted from images, such as gradient field HOG descriptor [Hu and Collomosse, 2013] and Learned Key Shapes (LKS) [Saavedra et al., 2015]. As for the deep learning based methods, Yu et al. [2017] first adopted CNN to learn better feature representation for sketches and images. Besides, siamese architecture [Qi et al., 2016] achieves a better metric of retrieval by minimizing the loss function for samples from the same category and maximizing the loss function for samples from different categories. ## 2.2 Zero-Shot Learning Existing zero-shot approaches can be classified into two categories: embedding-based and generative-based approaches. In the first category, some approaches learn non-linear multimodal embedding [Akata et al., 2015; Xian et al., 2016]. As for generative-based approaches, Wei et al. [2019] utilize a generator to synthesize unseen features and learn the semantic relations in a common space. It's worth noting that, most of ZSL methods utilize auxiliary information, e.g., a textbased embedding [Mikolov et al., 2013] or a hierarchical embedding [Miller, 1995] for label embedding, which is beneficial to knowledge transfer. ## 2.3 Zero-Shot Sketch-Based Image Retrieval The first work [Shen et al., 2018] of ZS-SBIR utilizes a multi-modal learning network to mitigate heterogeneity between two different modalities. The recent work SEMPCYC [Dutta and Akata, 2019] proposes a paired cycleconsistent generative model based on semantically alignment, which maintains a cycle consistency that only requires supervision at category level. Besides, SAKE [Liu et al., 2019; Kodirov et al., 2017] proposes a teacher-student network to maximally preserving previously acquired knowledge to reduce the domain gap between the seen source domain and unseen target domain. ## 3 Methodology ## 3.1 ProblemDefinition We first provide a formal definition of the ZS-SBIR task. Let Dtr = {X seen , S seen , Y seen } be a training set that contains sketch-image pairs X seen = {xske i , ximg i }Ns i=1 , semantic embedding S seen = {sseen i }Ns i=1 , and category labels Y seen = {yseen i }Ns i=1 with Ns samples. The test set is denoted as Dte = {X un , Y un } that contains sketches X un = {xun i }Nu i=1 and category labels Y un = {yun i }Nu i=1 with Nu samples, which satisfies the zero-shot setting Y seen ∩ Y un = ∅. During the test, given an unseen sketch xun i in X un , the objective is to retrieve corresponding natural images from the image gallery which contain all images including training images. The architecture of our proposed PDFD is illustrated in Figure 2, which contains two branches for sketch and image, respectively. Each branch first decomposes visual features into domain features and semantic features. Subsequently, decoders with shared parameters are trained to project semantic features into retrieval features for subsequent ZS-SBIR task. ## 3.2 Visual Features Decomposition In zero-shot learning, it is important to provide knowledge supervision when learning semantic features. Our proposed PDFD utilizes text-based embedding and hierarchical embedding to provide such supervision. ## Semantic Knowledge Embedding In PDFD, we adopt two widely-used text-based embedding, i.e., Word2Vec [Mikolov et al., 2013] and GloVe [Pennington et al., 2014] to obtain text representations. As for hierarchical embedding in PDFD, the similarity between words is calculated in WordNet 1 with path similarity and JiangConrath [Jiang and Conrath, 1997] similarity. ## Semantic Features As illustrated in Figure 2, each branch has a semantic encoder Eθsem , common discriminator Dθdis , and semantic embedding that combine text-based embedding and hierarchical embedding. Given a training sketch-image pair, their visual features are extracted from VGG16 [Simonyan and Zisserman, 2014] network pre-trained on ImageNet [Deng et al., 2009] dataset (before the last pooling layer). Then the semantic features are learned in an adversarial fashion, which means that the learned semantic features are expected to be as similar as the semantic embedding by 'fooling' the discriminator Dθdis . Specifically, the objective can be formulated as: where xske , ximg , sseen , Eθsem (·) and Dθdis (·) denote the sketch features, image features, semantic embedding, semantic generation function and discriminator function, respectively. Besides, the semantic generation function and discriminator function are parameterized by θsem and θdis. Here, Eθsem minimizes the objective against an opponent Dθdis that tries to maximize it. ## Domain Features Since semantic features and domain features are separated, we argue that semantic, image-domain features and sketchdomain features should be distinguished from each other. Thus, we categorize these three kinds of features into three different modalities. Here, the domain encoder Eθdom is adopted to attain the domain features with the constraint of the modality classifier. The modality classification loss can be where yske and yimg are labels indicating whether the corresponding features belong to sketch and image. Moreover, xdom ske , and xdom img denote the domain features from sketch and image branch respectively. They can be formulated as: It is worth noting that the domain features also have been constrained to cross-reconstruction loss, which will be introduced in Section 3.3. 1 https://wordnet.princeton.edu/ ## 3.3 Retrieval Features Generation After learning semantic features and domain features, PDFD generates retrieval features under two kinds of constraints. ## Classification Constraint It should be noted that the semantic features learned from two branches are only constrained by adversarial loss, which only ensures that the semantic features possess semantic knowledge. However, it can not guarantee the features to be classdiscriminative. Therefore, category classifier is introduced after the two branches. The category classification loss can be formulated as: where y is the category label of xske and ximg . Moreover, xsem ske , xsem img , xret ske and xret img denote the semantic features and retrieval features generated from sketch and image branch respectively. The generation of retrieval features can be formulated as: where Dθret (·) is the generation function of retrieval features. On the other hand, retrieval features should be domainindependent, such that they ought to be classified the semantic modality. We adopt the same modality classifier used in generation of domain features to ensure that, where the modality classification loss is written as: where ysem is the label indicating whether the corresponding features belong to semantic modality. ## Cross Reconstruction Constraint To ensure learning clean and complete retrieval features, we argue that such features should reconstruct the original sketch/image visual features by combined with sketch/imagedomain features. To this end, the cross-reconstruction loss is introduced to ensure the reconstructed features are similar to the original features. These reconstructed features are formulated as: where Rθ′ske (·) and Rθ′img (·) denote the reconstruction function on the sketch branch and image branch, respectively. Besides, ˜xske 1 and ˜xske 2 denote the reconstructed sketch features; ˜ximg 1 and ˜ximg 2 denote the reconstructed image features. Furthermore, the cross-reconstruction losses in sketch and image branch are written as: The total cross-reconstruction loss can be formulated as: ## 3.4 Objective and Optimization Since the modality loss is constrained both on domain features and semantic features, we can formulate the total modality loss as: Finally, the full objective of our proposed PDFD is: where λadv, λccls, λrec and λmcls are coefficients for balancing the overall performance. ## 4 Experiment ## 4.1 Datasets and Setup There are two widely-used large-scale sketch datasets Sketchy [Sangkloy et al., 2016] and TU-Berlin [Eitz et al., 2012] for ZS-SBIR. Sketchy originally consists of 75,479 sketches and 12,500 images from 125 categories. Liu et al. [2017a] extended the image gallery by collecting extra 60,502 images from ImageNet [Deng et al., 2009] dataset, such that the total number of images is 73,002 in the extended version. TU-Berlin originally consists of 20,000 unique free-hand sketches evenly distributed over 250 object categories. Compared to Sketchy, TU-Berlin only has category-level matches rather than instance-level matches. Following the same zero-shot data partitioning in SEMPCYC [Dutta and Akata, 2019], we also follow the same evaluation criterion in most previous works [Dutta and Akata, 2019; Shen et al., 2018] in terms of mean average precision (mAP@all) and precision considering the top 100 (Prec@100) retrievals. ## 4.2 Implementation Details PDFD is trained with Adam [Kingma and Ba, 2014] optimizer on PyTorch with an initial learning rate lr = 0.0001, β1 = 0.5, β2 = 0.99. The input size of the image/sketch is 224×224. We use the grid search method to select the best coefficients, which are λadv = 1.0, λrec = 1.0, λmcls = 1.0, λccls = 0.01 when training on Sketch and λadv = 1.0, λrec = 0.5, λmcls = 0.4, λccls = 0.4 when training on TUBerlin. VGG16 [Simonyan and Zisserman, 2014] pre-trained on ImageNet is adopted as a feature extractor. The word textbased embedding [Mikolov et al., 2013] is adopted to extract 300-dimensional word vectors. Under the zero-shot setting, we only consider the seen classes when constructing the hierarchy embedding [Miller, 1995] for obtaining the class embedding. Therefore, the hierarchical embedding for Sketchy and TU-Berlin respectively contain 354 and 664 nodes. | Methods | Feature - Dimension | Sketchy - mAP@all | Sketchy - Prec@100 | TU-Berlin - mAP@all | TU-Berlin - Prec@100 | |------------------------------------------|-----------------------|---------------------|----------------------|-----------------------|------------------------| | Siamese CNN [Qi et al., 2016] | 64 | 0.132 | 0.175 | 0.109 | 0.141 | | SaN [Yu et al., 2017] | 512 | 0.115 | 0.125 | 0.089 | 0.108 | | GN Triplett [Sangkloy et al., 2016] | 1024 | 0.204 | 0.296 | 0.175 | 0.253 | | SBIR 3D Shape [Wang et al., 2015] | 64 | 0.067 | 0.078 | 0.054 | 0.067 | | DSH (binary) [Liu et al., 2017a] | 64 | 0.171 | 0.231 | 0.129 | 0.189 | | GDH (binary) [Zhang et al., 2018] | 64 | 0.187 | 0.259 | 0.135 | 0.212 | | DeViSE [Frome et al., 2013] | 300 | 0.067 | 0.077 | 0.059 | 0.071 | | ZSL JLSE [Zhang and Saligrama, 2016] | 100 | 0.131 | 0.185 | 0.109 | 0.155 | | SAE [Kodirov et al., 2017] | 100 | 0.216 | 0.293 | 0.167 | 0.221 | | ZSH (binary) [Yang et al., 2016] | 64 | 0.159 | 0.214 | 0.141 | 0.171 | | ZSIH (binary) [Shen et al., 2018] | 64 | 0.258 | 0.342 | 0.223 | 0.294 | | CVAE [Kiran Yelamarthi et al., 2018] | 4096 | 0.196 | 0.284 | 0.005 | 0.001 | | SEM-PCYC [Dutta and Akata, 2019] | 64 | 0.349 | 0.463 | 0.297 | 0.426 | | SEM-PCYC(binary) [Dutta and Akata, 2019] | 64 | 0.344 | 0.399 | 0.293 | 0.392 | | ZS-SBIR SAKE(binary) [Liu et al., 2019] | 64 | 0.364 | 0.487 | 0.359 | 0.481 | | CSDB [Dutta and Biswas, 2019] | 64 | 0.484 | 0.375 | 0.355 | 0.254 | | PDFD (ours) | 64 | 0.623 | 0.726 | 0.460 | 0.595 | | PDFD (ours binary) | 64 | 0.638 | 0.755 | 0.386 | 0.542 | | SAKE [Liu et al., 2019] | 512 | 0.547 | 0.692 | 0.475 | 0.599 | | PDFD (ours) | 512 | 0.661 | 0.781 | 0.483 | 0.600 | Table 1: ZS-SBIR performance of our proposed PDFD compared with existing SBIR, ZSL and ZS-SBIR approaches. ![Image](https://ijcai.org/proceedings/2020/fulltext/137/image_000002_1dc2302303f0d2c45aee3ec8af2368b7ee013128c87bf5f807301de8b71350e5.png) (a) Retrieval results on Sketchy. (b) Retrieval results on TU-Berlin. Figure 3: The top 10 images retrieved by our PDFD on the two datasets. The red borders indicate mis-retrieved images. ## 4.3 Comparison with Peer Methods Apart from ZS-SBIR methods, some existing SBIR and ZSL approaches are also involved in retrieval comparison. The performances of all the comparisons are shown in Table 1, where we can observe that most ZS-SBIR methods outperform SBIR and ZSL methods while GN Triplet [Sangkloy et al., 2016] and SAE [Kodirov et al., 2017] reach the best performance in SBIR and ZSL, respectively. The main reason is that SBIR and ZSL methods are unable to take both domain gap and knowledge transfer into consideration. Therefore, the ZS-SBIR methods have better performance as they possess both the ability of reducing the domain gap and transferring the semantic knowledge. Due to the larger number of classes in TU-Berlin, all involved methods perform relatively worse on this dataset compared with Sketchy. Most of the ZS-SBIR methods are conducted to retrieve based on 64-dimensional features, so PDFD generates 64dimensional retrieval features for retrieval and outperforms the best competitor [Dutta and Biswas, 2019] by more than 13% on Sketchy and 10% on TU-Berlin. However, the SAKE [Liu et al., 2019] adopts the 512-dimensional features for retrieval, and then applies iterative quantization algorithm [Liu et al., 2017b] to obtain the 64-dimensional binary codes. For a fair comparison, PDFD also obtains 512-dimensional retrieval features and 64-dimensional binary codes. The result shows that our model significantly outperforms SAKE by around 11% on Sketchy and 1% on TU-Berlin when adopting 512-dimensional retrieval features. Table 2: The mAP@all results of PDFD using different semantic embedding and their combinations for ZS-SBIR. | Text-based embedding - Glove | Text-based embedding - Word2Vector | Hierarchical embedding - Path | Hierarchical embedding - Ji-Cn [Jiang and Conrath, 1997] | Sketchy | TU-Berlin | |--------------------------------|--------------------------------------|---------------------------------|------------------------------------------------------------|-------------------------------------------------|-------------------------------------------------| | ✓ ✓ ✓ | ✓ ✓ ✓ | ✓ ✓ ✓ | ✓ ✓ ✓ | 0.583 0.584 0.603 0.603 0.615 0.615 0.622 0.623 | 0.387 0.388 0.392 0.393 0.447 0.460 0.447 0.458 | Table 3: The mAP@all results of ablation study on our PDFD with several baselines for ZS-SBIR. | # | Description | Sketchy | TU-Berlin | |-----|---------------------------------------------------------------|-----------|-------------| | 1 | Baseline | 0.377 | 0.338 | | 2 | Baseline + Progressive (Lccls) | 0.481 | 0.374 | | 3 | Baseline + Progressive (Lccls) + Decomposition (Lmcls) | 0.510 | 0.396 | | 4 | Baseline + Progressive (Lccls) + Decomposition (Lrec) | 0.613 | 0.449 | | 5 | Baseline + Progressive (Lccls) + Decomposition (Lmcls + Lrec) | 0.623 | 0.460 | All of these demonstrate the effectiveness of PDFD for domain gap reduction and semantic knowledge transfer. The retrieved images for sketches are shown in Figure 3. The red borders indicate the incorrectly retrieved images. ## 4.4 Effect of Semantic Knowledge Embedding Since different types of semantic embedding have different impact on performance, we analyze the effects of different semantic embedding as well as different combinations of them on retrieval performance based on 64-dimensional retrieval features. Table 2 shows the quantitative results. As we can see, the combination of Word2Vec and Jiang-Conrath [Jiang and Conrath, 1997] hierarchical similarity reaches the highest mAP of 62.3% on Sketchy, while on TU Berlin dataset, the combination of Glove and Jiang-Conrath reaches the highest mAP of 46.0%. Note that we adopt the same embedding setting for all ablation studies. ## 4.5 Ablation Study Five ablation studies are conducted to validate the effectiveness of our proposed PDFD as exhibited in Table 3, which are: 1) A baseline that simultaneously projects visual features and semantic supervision into a low-dimensional common space; 2) Adding our progressive projection strategy to the baseline that first learns semantic features with the original semantic supervision, and then projects them as retrieval features under the category classification loss Lccls; 3) Further adding our proposed feature decomposition to attain domainindependent retrieval features under the modality classification loss Lmcls; 4) Replacing the modality classification loss Lmcls in "3)" with Lrec to validate the effectiveness of crossreconstruction; 5) Full PDFD model. Moreover, all the retrieval features in ablation studies are 64-dimensional. As shown in Table 3, our full model outperforms all baselines. The progressive projection strategy in "2)" improves baseline performance by around 10% on Sketchy and 4% on TU-Berlin, as this strategy benefits semantic knowledge transfer. By further decomposing visual features into semantic features and domain features under the modality classification loss Lmcls, we can derive domain-independent retrieval features and improve the cross-modal retrieval performance. Moreover, the proposed cross-reconstruction loss Lrec encourages learning retrieval features with clean and complete semantic information, which improves the performance by a large margin. Finally, with all proposed modules, the full model reaches the highest mAP@all of 62.3% on Sketchy and mAP@all of 46.0% on TU-Berlin. ## 5 Conclusion We have presented a novel progressive domain-independent feature decomposition network to address the problem of ZSSBIR more effectively. On one hand, a progressive projection strategy is exploited to preserve the semantic information with the strong supervision of original semantic knowledge for learning semantic features. On the other hand, the cross-reconstruction loss is imposed to reduce the domain gap by ensuring that the retrieval features capture clean and complete semantic information. Experiments on two large-scale datasets show that our proposed PDFD significantly outperforms existing state-of-the-art methods in ZS-SBIR task. ## Acknowledgments This work is supported in part by the Key R&D ProgramThe Key Industry Innovation Chain of Shaanxi under Grant 2019ZDLGY03-02-01, and in part by the National Key R&D Program of China under Grant 2017YFE0104100 and 2016YFE0200400. ## References 1. [Akata et al., 2015] Zeynep Akata, Florent Perronnin, Zaid Harchaoui, and Cordelia Schmid. Label-embedding for image classification. IEEE Trans. Pattern Anal. Mach. In- tell., 38(7):1425-1438, 2015. 2. [Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009. 3. [Dutta and Akata, 2019] Anjan Dutta and Zeynep Akata. Se- mantically tied paired cycle consistency for zero-shot sketch-based image retrieval. In CVPR, pages 5089-5098, 2019. 4. [Dutta and Biswas, 2019] Titir Dutta and Soma Biswas. Style-guided zero-shot sketch-based image retrieval. In BMVC, 2019. 5. [Eitz et al., 2012] Mathias Eitz, James Hays, and Marc Alex- a. How do humans sketch objects? ACM Trans. Graph., 31(4):44-1, 2012. 6. [Frome et al., 2013] Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc'Aurelio Ranzato, and Tomas Mikolov. Devise: A deep visual-semantic em- bedding model. In NeurIPS, pages 2121-2129, 2013. 7. [Hu and Collomosse, 2013] Rui Hu and John Collomosse. A performance evaluation of gradient field hog descriptor for sketch based image retrieval. Comput. Vis. Image Under- stand., 117(7):790-806, 2013. 8. [Jiang and Conrath, 1997] Jay J Jiang and David W Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. arXiv preprint cmp-lg/9709008, 1997. 9. [Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arX- iv:1412.6980, 2014. 10. [Kiran Yelamarthi et al., 2018] Sasi Kiran Yelamarthi, Shiv- a Krishna Reddy, Ashish Mishra, and Anurag Mittal. A zero-shot framework for sketch based image retrieval. In ECCV, pages 300-317, 2018. 11. [Kodirov et al., 2017] Elyor Kodirov, Tao Xiang, and Shao- gang Gong. Semantic autoencoder for zero-shot learning. In CVPR, pages 3174-3183, 2017. 12. [Liu et al., 2017a] Li Liu, Fumin Shen, Yuming Shen, Xi- anglong Liu, and Ling Shao. Deep sketch hashing: Fast free-hand sketch-based image retrieval. In CVPR, pages 2862-2871, 2017. 13. [Liu et al., 2017b] Li Liu, Mengyang Yu, and Ling Shao. Learning short binary codes for large-scale image re- trieval. IEEE Trans. Image Process., 26(3):1289-1299, 2017. 14. [Liu et al., 2019] Qing Liu, Lingxi Xie, Huiyu Wang, and Alan L Yuille. Semantic-aware knowledge preservation for zero-shot sketch-based image retrieval. In ICCV, pages 3662-3671, 2019. 15. [Mikolov et al., 2013] Tomas Mikolov, Kai Chen, Greg Cor- rado, and Jeffrey Dean. Efficient estimation of word rep- resentations in vector space. arXiv:1301.3781, 2013. 16. [Miller, 1995] George A Miller. Wordnet: a lexical database for english. ACM Commun., 38(11):39-41, 1995. 17. [Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014. 18. [Qi et al., 2016] Yonggang Qi, Yi-Zhe Song, Honggang Zhang, and Jun Liu. Sketch-based image retrieval vi- a siamese convolutional neural network. In ICIP, pages 2460-2464, 2016. 19. [Saavedra et al., 2015] Jose M Saavedra, Juan Manuel Bar- rios, and S Orand. Sketch based image retrieval using learned keyshapes(lks). In BMVC, volume 1, page 7, 2015. 20. [Sangkloy et al., 2016] Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, and James Hays. The sketchy database: learning to retrieve badly drawn bunnies. ACM Trans. Graph., 35(4):119, 2016. 21. [Shen et al., 2018] Yuming Shen, Li Liu, Fumin Shen, and Ling Shao. Zero-shot sketch-image hashing. In CVPR, pages 3598-3607, 2018. 22. [Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556, 2014. 23. [Wang et al., 2015] Meng Wang, Chaokun Wang, Jeffrey Xu Yu, and Jun Zhang. Community detection in social net- works: an in-depth benchmarking study with a procedure- oriented framework. In VLDB, pages 998-1009, 2015. 24. [Wei et al., 2019] Kun Wei, Muli Yang, Hao Wang, Cheng Deng, and Xianglong Liu. Adversarial fine-grained com- position learning for unseen attribute-object recognition. In ICCV, pages 3741-3749, 2019. 25. [Xian et al., 2016] Yongqin Xian, Zeynep Akata, Gaurav Sharma, Quynh Nguyen, Matthias Hein, and Bern- t Schiele. Latent embeddings for zero-shot classification. In CVPR, pages 69-77, 2016. 26. [Yang et al., 2016] Yang Yang, Yadan Luo, Weilun Chen, Fumin Shen, Jie Shao, and Heng Tao Shen. Zero-shot hashing via transferring supervised knowledge. In ACM- MM, pages 1286-1295, 2016. 27. [Yu et al., 2017] Qian Yu, Yongxin Yang, Feng Liu, Yi-Zhe Song, Tao Xiang, and Timothy M Hospedales. Sketch- a-net: A deep neural network that beats humans. Int. J. Comput. Vis., 122(3):411-425, 2017. 28. [Zhang and Saligrama, 2016] Ziming Zhang and Venkatesh Saligrama. Zero-shot learning via joint latent similarity embedding. In CVPR, pages 6034-6042, 2016. 29. [Zhang et al., 2018] Jingyi Zhang, Fumin Shen, Li Liu, Fan Zhu, Mengyang Yu, Ling Shao, Heng Tao Shen, and Luc Van Gool. Generative domain-migration hashing for sketch-to-image retrieval. In ECCV, pages 297-314, 2018.