Disentangled Variational Autoencoder based Multi-Label Classification with Covariance-Aware Multivariate Probit Model

1 Introduction

Multi-label classification (MLC) concerns the simultaneous prediction of the presence and absence of multiple labels for each sample of a given sample set. Unlike in the conventional classification task, more than one label or target could be associated with each sample in MLC [Zhang and Zhou, 2013; Zhang et al., 2018]. This setting is important for the study of key biodiversity applications in computational sustainability [Gomes et al., 2019], e.g., concerning joint species distributions and wildlife mapping [Chen et al., 2017], as well as in a variety of other scenarios, such as protein site localization [Alazaidah et al., 2015] and drug side effects [Kuhn et al., 2015]. Furthermore, understanding the label correlations is also important. For instance, in biodiversity applications, species correlation modeling is critical to address core ecological concerns like interactions of species with each other, which could affect species monitoring, protection and policymaking [Evans et al., 2017].

∗Contact Author

1 Our code is available on https://github.com/JunwenBai/MPVAE

Some early work simply decomposes the multi-label classification into multiple single-label classification problems [Boutell et al., 2004]. Though these methods can be adapted from single-label predictors, they ignore the correlation among labels. To improve this, classifier chains [Read et al., 2009] stack the binary classifiers into a chain and reuse the outputs of previous classifiers as extra information to improve the prediction of the current label. Followup works extend the classifier chains to recurrent neural networks [Wang et al., 2016] to increase capacity and better model the label correlation. Label ordering is critical to these methods since long-term dependencies are typically weaker than shortterm dependencies. The model structure also restricts parallel computation. Another straightforward method is to find nearest neighbors in the feature space and assign labels to test samples by Bayesian inference [Zhang and Zhou, 2007; Chiang et al., 2012]. However, either the predefined metric space or the prior may heavily affect the model performance.

Latent embedding learning is a recent technique to match features and labels in the latent space. Pioneer studies [Yu et al., 2014; Chen and Lin, 2012; Bhatia et al., 2015] make low-rank assumptions of labels and features, and transform labels to label embeddings, by dimensionality reduction techniques such as canonical correlation analysis (CCA). Benefiting from the capacity of deep neural networks, more recent latent embedding methods for MLC employ neural networks to build and align the latent spaces for both labels and features [Yeh et al., 2017; Chen et al., 2019a]. The constraints in the conventional dimension reduction models are relaxed and embedded into the deep latent space. For example, C2AE relaxes the orthogonality constraint to the minimization of an ℓ2 distance. These embedding methods are believed to implicitly encode the label correlations in the embedding space.

Some other state-of-the-art models initiate the research on using graph neural networks (GNN) to explicitly encode the label correlations [Chen et al., 2019b; Lanchantin et al., 2019]. A graph neural network for labels can build dependencies among labels through learned or given edges between them. Though GNN brings a new way to embed the correlations, the number of stacked GNNs or the iterations of message passing may require extra effort to fine-tune.

We propose the Multivariate Probit Variational Autoencoder (MPVAE), which improves both the embedding space learning and label correlation encoding. In particular, (1) MPVAE learns probabilistic latent spaces for both labels and features, unlike most autoencoder (AE) based multi-label models. The probabilistic latent space learned by the VAE can provide three major advantages. First, it gives more control to the latent space [Chung et al., 2015]. In many AE models, one can often observe the label-encoder-decoder branch gives much better performance than feature-encoder-decoder branch. Imposing the VAE structure in the latent space helps balance the difficulty of the learning and aligning of the two subspaces. Second, smoothness in the latent space is often desired [Wu et al., 2018]. Probabilistic models like the VAE naturally bring smoothness on a local scale since the decoder decodes a sample rather than a specific embedding. Third, the VAE model and its variations learn representations with disentangled factors [van Steenkiste et al., 2019]. If both latent spaces for features and labels learn disentangled factors, not only is it helpful to aligning two spaces, but it is also beneficial for the decoding process. (2) MPVAE explicitly learns a shared covariance matrix to build dependencies among labels by adopting the Multivariate Probit (MP) probabilistic model, which is inspired by some recent work in joint distribution modeling [Chen et al., 2018]. The MP assumes an underlying latent multivariate Gaussian distribution. We show that the MP model is a simple and straightforward component of the overall probabilistic generative framework compared to other more complex models such as GNNs. More importantly, the MP model improves the prediction performance and provides the interpretability of the learned covariance matrix. By using the Cholesky decomposition and tSNE, we demonstrate visually the value of the learned covariance, in applications like species correlation modeling in the computational sustainability domain. (3) MPVAE is optimized with respect to a three-component loss function, which includes a Kullback-Leibler (KL) divergence component for jointly learning and aligning the label and feature embeddings and a cross-entropy and ranking loss for the multilabel prediction in the MP model. (4) We thoroughly test MPVAE with multiple public datasets on a variety of metrics. MPVAE outperforms (or is comparable to) other state-ofthe-art multi-label prediction models. We further illustrate MPVAE is still robust even if the training labels are noisy.

2 Other RelatedWork

Covariance matrices are commonly seen in non-deep multilabel classification models [Bi and Kwok, 2014; Zhang and Yeung, 2013]. Non-deep models often assume a matrixvariate normal distribution on features, weights or labels. Covariances thus play a key role in these models. But the scalability issue limits their applications in large-scale problems. However, these assumptions are not necessary in deep neural networks given their powerful expressiveness.

A recent success of combining deep learning and covariance based paradigms is the deep Multivariate Probit model (DMVP) [Chen et al., 2018]. It is a deep generalization of the classic Multivariate Probit model. An efficient sampling process was proposed in the paper to avoid the heavyduty Markov chain Monte Carlo (MCMC) sampling process. Though DMVP performs well on the joint likelihood measure, it lacks enough predictive power for presence-absence (0/1) classification. MPVAE makes one step further and introduces the cross-entropy loss as well as the ranking-loss under the Multivariate Probit paradigm.

Some prior work also applied deep generative models in multi-label classification. [Chu et al., 2018] proposes a deep sequential generative model. Though the model is effective in the setting of missing labels, the concern of unstable training of stacked generative models should not be overlooked.

3 Methods

3.1 Preliminaries

Let D denote the dataset {(xi, yi )}Ni=1 , where xi ∈ RS and yi ∈ {0, 1}L. xi is the feature vector and yi represents the presence (1) or absence (0) of targets.

Variational Autoencoder (VAE)

A VAE assumes a generative process for the observed datapoints X: P(X) = ∫ pθ(X|z; θ)P(z)dz, by introducing latent variables z. Since most z's contribute little to P(X), Monte Carlo sampling would be inefficient. We instead learn a function qφ(z|X) to approximate the intractable P(z|X) for efficient sampling. The KL divergence (D) between qφ(z|X) and P(z|X) is given by D(qφ(z|X)||P (z|X)) = Ez∼qφ [log qφ(z|X) - log P (z|X)]. Applying Bayes rule and transforming the equation yield: log P (X) - D[Q(z|X)||P (z|X)] = Ez∼qφ [log pθ(X|z)] - D[qφ(z|X)||P(z)]. The right hand side of the equation is the tractable evidence lower bound (ELBO) to maximize. P(z) is the prior, a standard multivariate normal distribution. The first term in the ELBO encourages the reconstruction of X and the second term penalizes the KL divergence between the approximate distribution and the prior to impose structure on the latent space. Both pθ and qφ can be parameterized by neural networks. With the help of the reparameterization trick, the whole model can be trained with back-propagation. VAE and its variations can learn disentangled factors by controlling the capacity of the information bottleneck. For example, β-VAE [Higgins et al., 2017] is a known disentangled VAE, which is able to learn abstract concepts like size and shape, only with a slight modification to the objective, Ez∼qφ [log pθ(X|z)] - βD[qφ(z|X)||P (z)]. MPVAE follows the structure of β-VAE with β = 1.1.

Multivariate Probit (MP) Model

Consider a single sample (x, y) where x is the input feature vector and y ∈ {0, 1}L is the label. The Multivariate Probit model introduces auxiliary latent variables y∗ ∈ RL, which follow a multivariate normal distribution N(xγ, Σ) where γ is the weight parameter and Σ is the covariance matrix. y is viewed as the indicator for whether y∗ is positive or not: yi = 1{y∗ i > 0}, i = 1, .., L. The probability of observing y is given by P(y|xγ, Σ) = ∫AL ... ∫A1 p(y∗ |xγ, Σ)dy∗ 1 ...dy∗L, where Aj = (-∞, 0] if yj = 0 or Aj = (0,∞) otherwise. p(·) is the probability density function of the normal distribution. The framework can be generalized to a deep model simply by replacing the mean xγ with a neural network f(x). In MPVAE, the mean is given by the decoder of VAE.

3.2 MPVAE

We propose Multivariate Probit Variational Autoencoder, a novel disentangled Variational Autoencoder based framework with covariance-aware Multivariate Probit model for MLC. The illustration of the framework is shown in Fig. 1. The whole model can be viewed as a two-stage generative process. The first stage maps the features and labels to Gaussian subspaces where the means and variances are learned by multi-layer perceptrons. The key task in this stage is to match the two subspaces. The second stage decodes the sample from each subspace and feeds the outputs into a Multivariate Probit module as the means. A global covariance matrix is learned separately. The final output of the second stage gives the predicted labels.

Learning and Aligning Probabilistic Subspaces

Given an input pair of feature vector and label (x, y) where x ∈ RS , y ∈ {0, 1}L, the feature encoder maps x to a Gaussian subspace N(µψ (x),Σψ (x)) and the label encoder maps y to another Gaussian subspace N(µφ (y),Σφ (y)). φ, ψ are trainable parameters in the encoders. µψ (x), µφ (y) ∈ Rd and Σψ (x),Σφ (y) ∈ Rd×d ≥0 , where d is the dimensionality of the latent space. zx, zy denote samples from each of these two distributions respectively.

If we only consider the label encoder-decoder branch (orange branch in Fig. 1), it models a standard (β-)VAE. The ELBO to optimize can be written as,

The issue with this label autoencoder is the lack of connections between x and z. Even if a good generative model is learned, prediction given x is impossible since the prior P(z) is unrelated to x. Our simple fix is to replace P(z) with a prior distribution dependent on x, qψ(z|x). That's where the feature encoder comes from. The feature encoder is also a neural network parameterized by learnable ψ. With the feature encoder, given the input x, we can sample from qψ(z|x), which is approximately equal to qθ(z|y). The challenges thereafter are twofold: a) how to learn ψ and b) how to align N(µψ (x), Σψ (x)) and N(µφ (y), Σφ (y)). We propose to extend the objective function to:

The first two terms will be handled by the Multivariate Probit model in the next subsection. The last term is simply the KL divergence between two multivariate normal distributions. Since both distributions have diagonal covariance matrices, we can derive the KL loss term for D[qφ(z|y)||qψ(z|x)]:

As in the conditional VAE, our implementation further adds an improvement to concatenate each of y, zy, zx with x, which basically does not affect the derivations above, but uses extra information for inference.

Reconstruction and Prediction

The Multivariate Probit (MP) is a classic latent variable model for data with presence-absence relationships. Unlike the typical softmax transformation in most deep methods, the model maps a sample from a multivariate normal distribution to its own cumulative distribution (CDF), to bound the output range within [0, 1]. The major drawback of the MP model is that the integration step is intractable for large-scale data and is usually approximated with MCMC. [Chen et al., 2018] provides an alternative parallelizable sampling process for estimating the CDF. GPUs can thus expedite the estimation. We adopt this sampling method in our model, and find it works well in MPVAE.

Label information is available for both the training and testing phases in [Chen et al., 2018]. However, as a predictive model, MPVAE does not have access to labels until the final predicted targets are given, when the loss can be computed between the ground-truth and predicted labels for training, or the predicted targets are directly given as the output during testing. In this case, we let MPVAE calculate the integral only w.r.t. the (0,∞) region for each target(dimension). If the corresponding target is present, the CDF should be close to 1. Otherwise, it should be near 0.

Sampling Process. The shared decoder reads the samples zx, zy, and outputs mx, my ∈ RL respectively as the means in the MP model. The covariance matrix Σg ∈ RL×L in the MP only models the dependencies among targets and is unrelated to the features. Thus Σg can be learned as a shared parameter. This stabilizes the training and helps interpret the label correlation (shown in experiments). Instead of directly sampling y∗ from N(m,Σg), we sample twice by decomposing Σg to V + Σr where V is a diagonal positive definite matrix and Σr is the residual. For 2 random variables w ∼ N(0, V ), s ∼ N(m,Σr), the difference (w - s) ∼ N(-m,Σg). Note that since V is diagonal, the different dimensions of w are independent. To estimate the probability of presence P(y∗ i ≥ 0|m, Σg), i ∈ [1, L],

where Φ is the CDF of a univariate standard normal distribution. Since V is simply a scaling factor, w.l.o.g., V can be set to identity I. Under this mechanism, P(y∗ i ≥ 0|m,Σg) for each i can be computed in parallel and independently with one sample s or the average of multiple samples. Note that with this derivation, Σr is learned rather than Σg. But they only differ by I. The 0/1 outputs given mx,my are denoted by ˆyf and ˆyl . If P(y∗ i ≥ 0|m,Σg) is higher than a certain threshold, ˆ yi gives 1. Otherwise, ˆ yi is set to 0. During testing, ˆyf is regarded as the final result.

Binary Cross Entropy (BCE) Loss in MP. With the efficient sampling scheme, the reconstruction losses Ez∼qφ [log pθ(y|z)] and Ez∼qψ [log pθ(y|z)] can be concretized. For a binary prediction task for each target, a Bernoulli likelihood assumption is valid, which leads to the binary cross entropy loss between the labels and predictions:

The last step is a Monte Carlo approximation. M is the preset number of samples. The log-sum-exp trick is applied for computation to avoid overflow issues. Since Ez∼qφ [log pθ(y|z)] is typically approximated by a single sample from qφ and minimization is preferred for training, the

Algorithm 1 Training MPVAE

Input: {(xi, yi )}Ni=1 , batch size b

• 1: for # of iterations do
• 2: Sample a minibatch {(x(k) , y(k) )}bk=1
• 3: Compute {(µψ (x(k) ),Σψ (x(k) )), (µφ (y(k) ),Σφ (y(k) ))}
• 4: Derive LKL by Eq (2)
• 5: Sample latent variables {z(k) x }, {z(k) y }
• 6: Decode z's and obtain {m(k) x }, {m(k) y }
• 7: Calculate the mean loss by Eq (5) for the batch
• 8: Compute gradients and update parameters with Adam
• 9: return φ, ψ, θ,Σr

objective function can be written as -Ez∼qφ [log pθ(y|z)] ≈ -log pθ(y|zy) = LCE(y, zy) where zy ∼ qφ(z|y). Similarly, -Ez∼qψ [log pθ(y|z)] ≈ - log pθ(y|zx) = LCE(y, zx) where zx ∼ qψ(z|x).

Ranking Loss in MP. Ranking loss [Zhang and Zhou, 2013] is also widely used in many multi-label tasks. It is a loss to measure the correlations between positive labels and negative labels. The idea behind the ranking loss is simple: the gap between the logits for positive and negative labels should be as large as possible. Suppose y is the groundtruth label set. Let y0 denote the set of indices of negative labels and y1 the positive labels. s denotes the sample from N(m,Σr), which depends on z. Thus the ranking loss on s can also be viewed as a loss on z. We define the ranking loss in MP as

The ranking loss can be defined for both zy and zx.

Entropy Loss in MP. Some multi-label datasets have very sparse positive labels even if there exist multiple labels for one feature. For example, in the mirflickr dataset, the positive label rate is as low as 12%. Therefore, for sparse datasets, we add an extra entropy loss (define pi = exp(Φ(si)) ∑Lj=1 exp(Φ(si)) ),

Entropy loss acts as a self-regularizer and only depends on the predicted values.

Overall Loss Function

Let Lprior = λ1LCE(y, zx) + λ2LRL(y, zx) + λ3LEnt(zx) and Lrecon = λ1LCE(y, zy) + λ2LRL(y, zy) + λ3LEnt(zy). Together with the KL loss LKL defined in the previous subsection, the overall loss function can be described as

Lprior encompasses the losses in the feature-encoder-decoder branch (blue branch) for the prior, while Lrecon encompasses the losses in the label-encoder-decoder branch (orange branch) for the reconstruction. λ1, λ2, λ3 control the weights of the three loss terms in each branch and are the same in each branch. β governs the information bottleneck in β-VAE. Both branches share the decoder and the Multivariate Probit module, which in turn helps and regularizes the learning of the latent subspace where z is embedded. The whole model can be trained end-to-end through back-propagation with Adam [Kingma and Ba, 2015] (see Alg. 1).

Interpretability of Σg

One thing special in MPVAE compared to other multi-label prediction methods is the global parameter Σg (i.e. Σr + I). Suppose each target/label can be represented by a vector vi, i ∈ [1,L], the correlations can be captured by the inner product between vi and vj. If the covariance matrix Σg indeed contains such correlation, by Cholesky decomposition Σg = V V T , each row of V could be regarded as vi. By some dimension reduction tricks like t-SNE, the vectors of similar targets should be close. For illustration purposes, we show in the experiments section, using a real-world dataset (eBird), that Σg does capture such information.

4 Experiments

4.1 Datasets

MPVAE is validated on 9 real-world datasets from a variety of fields including ecology, biology, images, texts, etc. The datasets are eBird [Munson et al., 2011], North American fish [Morley et al., 2018], mirflickr [Huiskes and Lew, 2008], NUS-WIDE 2 [Chua et al., 2009], yeast [Nakai and Kanehisa, 1992], scene [Boutell et al., 2004], sider [Kuhn et al., 2016], bibtex [Katakis et al., 2008], and delicious [Tsoumakas et al., 2008]. eBird is a crowd-sourced bird presence-absence dataset collected from birders' observations. North American fish (fish) is a fish distribution dataset collected from the trawlers in the North Atlantic. yeast is a biology database of the protein localization sites and sider is another database of drug side-effects. mirflickr, NUS-WIDE (nuswide), and scene datasets are from the image domain. Finally, the bibtex dataset contains a large number of BibTex files online and the delicious dataset contains web bookmarks.

The datasets represent a large variety of different scales. The dimensionality varies from 15 (eBird) to 1836 (bibtex). The number of labels ranges from 6 (scene) to 983 (delicious). The size of the datasets could be as high as 200,000 (nuswide), or as low as 1427 (sider). Most datasets are available on a public website 3 . The rest can be found in the related papers. If a dataset has been split a priori, we follow those divisions. Otherwise, we separate the dataset into training (80%), validation (10%) and testing (10%). The datasets are also preprocessed to fit the requirements of the input formats for the different methods. For example, we expand the input features with word embeddings for LaMP.

4.2 Implementation andModel Comparison

The encoders and decoder of MPVAE are parameterized by 3-layer fully connected neural networks with latent dimensionalities 512 and 256. The compared models share the same neural network structure for fair comparison, in cases where neural networks are used. The activation function in the neural networks is set to ReLU. Σr is a shared learnable parameter of size L × L. By default, we set β = 1.1, λ1 = λ3 = 0.5, λ2 = 10.0. These default hyperparameter values are inherited from the existing well-trained DMVP [Chen et al., 2018] and β-VAE [Higgins et al., 2017] models. We achieve the best performance for our own model in the neighborhood of this default set of parameters via grid search. We also use grid search to find the best learning rate, learning rate decay ratio and dropout ratio hyperparameters. Note that since not all datasets have sparse labels, λ3 could be set to 0 or close to 0 in such scenarios. In practice, we found larger λ2 gives better performance because the average ranking loss of multiple samples could provide good guidance for training the model. β is the tradeoff between the capacity of the information bottleneck and the learnability of the decoder. In our experiments, the best values for β are in the vicinity of 1.1.

2 We only use the 128-d cVALD features as the input.

3 http://mulan.sourceforge.net/datasets-mlc.html

MPVAE is compared with 6 other state-of-the-art methods for multi-label prediction. MLKNN [Zhang and Zhou, 2007] is a nearest neighbor based algorithm. Bayesian inference is applied for testing. MLARAM [Benites and Sapozhnikova, 2015] is a scalable extension to the adaptive resonance associative map neural network designed for large-scale multilabel classification. SLEEC [Bhatia et al., 2015] learns a small ensemble of embeddings preserving local distances. It makes low-rank assumptions and can be improved with neural networks (implemented for comparison). C2AE [Yeh et al., 2017] is a recently proposed approach that learns a deep latent space through an autoencoder structure. Features and labels are encoded through deep neural networks into a latent space, where the latent embeddings for features and labels are associated by deep canonical correlation analysis (DCCA). DMVP [Chen et al., 2018] is proposed for joint likelihood modeling but can be used for prediction if the trained model follows the sampling process in section 3.3 in the test phase. LaMP [Lanchantin et al., 2019] is the state-of-the-art GNNbased model for multi-label prediction. It encodes the correlations among labels to a GNN, and predicts unseen instances with the trained GNN.

The major evaluation metrics for multi-label predictions are example-based F1 (example-F1), micro-averaged F1 (micro-F1) and macro-averaged F1 (macro-F1) scores.

Example-F1 measures the proportion of true posinoise level tive predictions among the aggregation of the positive ground-truth labels and positive predicted labels: 1 Nt ∑Nt i=1 ∑Lj=1 2yij ˆ yij ∑Lj=1 yij +∑Lj=1 ˆ yij , where Nt is the number of test samples, yij is the j-th actual label of test sample i and ˆ yij is the j-th predicted label of test sample i. The top table in Table 1 shows the performances of different methods on all the datasets. Each F1 score is the average of 3 runs (same for the numbers in other tables and figures). MPVAE outperforms other methods on this metric. On average, MPVAE yields a 6% improvement compared to LaMP and a 9% improvement compared to C2AE. Micro-F1 computes the average F1 scores over all samples:

Figure 3: Micro-F1 scores of the methods under different noise levels. Top: fish dataset. Bottom: mirflickr dataset.

∑Lj=1 ∑Nt i=1 2yij ˆ yij ∑Lj=1 ∑Nt i=1 [2yij ˆ yij +(1-yij )ˆ yij +yij (1-ˆ yij )] . The results are given in the middle table in Table 1. MPVAE is only slightly worse than DMVP on bibtex, but compared to DMVP, MPVAE performs better by 2.5% on average. The third F1-related metric is the macro-F1 score, which is the averaged F1 score over all labels: 1L ∑Li=1 ∑Nt i=1 2yij ˆ yij ∑Nt i=1 [2yij ˆ yij +(1-yij )ˆ yij +yij (1-ˆ yij )] . Results on the bottom table in Table 1 illustrate that MPVAE outperforms other methods except on delicious (second best).

Besides the 3 most commonly used metrics, we test MPVAE on 2 other metrics: Hamming accuracy, and Precision@K. Hamming accuracy measures how many labels are predicted correctly: 1 Nt 1L ∑Nt i=1 ∑Lj=1 1[yij = ˆ yij ], regardless of pos/neg. Accuracies are collected in Table 2. The other metric Precision@K is defined as the percentage of correctly predicted labels in the top-K predictions. MPVAE is validated on two datasets eBird and sider, w.r.t. Precision@K (Fig. 2). The definition, implementation and threshold selection on the validation set for the evaluation metrics follow the paper [Lanchantin et al., 2019].

4.3 Noisy Labels

Noisy labels are quite common in real-world datasets. For example, in trawl survey data of fish, the raw collected presence or absence of species might be misrecorded [Carton et al., 2018]. Though the datasets we use have been cleaned and calibrated, the noisy setting can be reproduced by randomly flipping the labels in the training data. We tested all the methods on fish and mirflickr w.r.t. 4 noise levels: 10%, 20%, 30% and 40%. The comparisons are demonstrated in Fig. 3. As the noise level increases, MPVAE is still the most robust one. This is because when the latent dimensionality of VAE is relatively small, the model is forced to focus on the strong patterns and ignore the noise. The global covariance matrix also helps with the robustness. But as the noise level reaches 30% and beyond, all the methods perform much worse since the noise affects the whole distribution.

4.4 Interpreting the Covariance Σg

We validate the interpretability of Σg on eBird dataset. As we mentioned in section 3.3, Σg can be decomposed as V V T . We regard each row of V as vi and plot these vectors using t-SNE (see Fig. 4). One can observe that birds in the same category are clustered together. Similar clusters are also close to each other; e.g., water birds and birds near wetlands have similar embeddings. Since forest and pasture birds are the most commonly seen birds, it's not surprising that they spread across the plot. In contrast, the embeddings of rare raptors are close together. The bird categories and habits are collected from experts and professional websites 4 .

5 Conclusion

In this paper, we propose a disentangled Variational Autoencoder based framework incorporating covariance-aware Multivariate Probit model (MPVAE) for multi-label prediction. MPVAE comprises a feature encoder, a label encoder, a shared decoder and a Multivariate Probit model. Encoders are learned for the features and labels respectively to map them to a probabilistic subspace. The samples from the subspaces are decoded under the Multivariate Probit model to give the prediction. The disentangled β-VAE module improves the label embedding learning as well as feature embedding learning. The Multivariate Probit module provides a simple and convenient way to capture the label correlations. More importantly, we claim that the learned covariance matrix in the MP model is interpretable as shown in a real-wolrd dataset. MPVAE performs favorably against other state-of-the-art methods on 9 public datasets and remains effective under noisy settings, which verifies the usefulness and robustness of our proposed model.

4 https://ebird.org/home

Acknowledgments

This work is supported by National Science Foundation awards OIA-1936950 and CCF-1522054. We also want to thank the Cornell Lab of Ornithology and Gulf of Maine Research Institute for providing data, resources and advice.