Improving Tandem Mass Spectra Analysis with Hierarchical Learning

1 Introduction

Identifying proteins in complex biological samples is an elementary task in medicine and biology, such as analysis of components in the blood. Tandem mass spectrometry (MS/MS) is widely employed to accomplish this task. At a general MS/MS experiment, a collection of spectra is created in a particular order, each of which is representative of a protein subsequence called a peptide. The pair consisting of a matched peptide sequence and spectrum is considered as a peptide-spectrum match (PSM). Accurately identifying the peptide sequences responsible for each experimental spectrum becomes a challenging task in Proteomics.

Recent advances in deep learning technology have substantially improved the performance of peptide identification. The essential practice of neural peptide sequencing models follow encoder-decoder paradigm [Tran et al., 2017; Tran et al., 2019; Qiao et al., 2019]. In between, convolutional neural network (CNN) is utilized to encode an input spectrum and recurrent neural network (RNN) is adopted as a sequence decoder to produce the entire peptide sequence, one amino acid at each time step. Most of these peptide sequencing approaches are trained by maximizing the likelihood of each ground-truth amino acid based on previously generated amino acids and the spectrum with backpropagation.

However, there exist two major problems in these peptide sequence identification methods: (1) It is unreliable for the sequence decoders to generate amino acid one by one as a peptide sequence from left to right. Since the fragments of the peptide are more likely to occur in the middle position in a biological experiment, which results in a lower abundance of effective signal peaks on both sides of the spectrum and easy to be concealed by noise peaks [Li et al., 2005]. On the other hand, the peptide sequencing process is accomplished by a greedy search or with a beam search, which predicts the next amino acid according to local maximum occurrence probability. There is a case that some total matched peptide sequences may be filtered at early steps which are with low probability to be recognized in similar noise peaks by using the local maximum probability network alone. Such a top ranking-based approach assumes that the log-probability of every amino acid in a match sequence must be among top choices. Actually, this is not necessarily true. (2) Despite involving two different modalities in peptide sequencing, former approaches seldom explore the interactions between CNN and RNN structure. A common method is directly feeding the spectrum feature from CNN into the RNN as the initial node [Tran et al., 2017]. However, such a naive method treats features in the experimental spectrum the same and ignores the major influence of local features when generating each amino acid. Uncorrelated global spectrum information may interfere with current peptide sequence generation and severely limits the capacity of complex reasoning.

Considering the great challenge of generating peptide sequences from left to right in one stage, we propose a novel hierarchical multi-stage framework, namely DeepTag, that generates peptide sequences with the help of guiding tags [Tabb et al., 2003], which refer to high-score peptide subsequences, will be expanded in later decoders. Technically, our model consists of a spectrum encoder and a sequence of Transformer-based peptide decoders that gradually expand peptide sequences in two sides. To refine the information of decoder at the input time, we adopt a refining module, which can fuse historical subsequence information and spectrum information to conduct the sequence generation. Different from [Tran et al., 2017], our model utilize cross-modality refining mechanism [Vaswani et al., 2017] to weight spectrum features based on the preceding output of Transformer decoder layer. Furthermore, inspired by the recent work [Zhang et al., 2017], we design a similar RL-based training method, but extend it from one-stage to our multi-stage framework, where rewards are incorporated at each stage as intermediate supervision. Figure 1 illustrates our proposed hierarchical framework, which consists of three peptide sequence decoder networks. Specifically, the first Transformer decoder generates the guiding tag for the observed spectrum, and the subsequent Transformer decoders serve as sequence extension. At each stage in our DeepTag, refined spectrum features and hidden vector produced by the preceding decoding stage are adopted as inputs to the subsequent stage.

Our contributions are summarized as follows:

• We introduce a hierarchical multi-stage framework to improve the peptide identification from mass spectra, which iteratively expands the peptide sequences layer by layer based on high-confidence guiding tags.
• We present a new multi-layer cross-modality refining mechanism to refine the spectrum features before feeding to the sequence decoders, which significantly boosts the performance of peptide sequencing.
• We incorporate a modified RL-based fine-tune technique that can optimize the multi-stage model with the normalized intermediate rewards.

2 Background

Tandem mass spectrometry is the key technology for mixed sample detection and biology pathological research. It has numerous successful applications in medical, biology, and pharmacy [Craig and Beavis, 2004]. In practical MS/MS experiments, the digesting enzyme is first employed to cleave proteins into peptides. Next, the generated peptides are successively imported to the mass spectrometry in two rounds. In the first round, the mass and charge of enact peptide, called precursor mass and precursor charge, are measured. In the second round, the peptides are further fragmented and a sequence of mass spectra are provided, which corresponds to a small part of peptide [Li et al., 2005]. Specifically, each output mass spectrum is a set of ( mz , intensity) pairs where mz denotes the mass-to-charge ratio and intensity value is the abundance of these ions.

In this study, we aim to reconstruct the amino acid sequence of a peptide on the basis of the given spectrum. Nevertheless, the major challenge lies that: (1) a large number of candidate combinations lead to high computational complexity. Since there will be exponential situations to be considered as the candidate prediction in an extreme case. (2) peptide fragmentation produces multiple types of ions that hold quite different intensity values. The division rule is critical but remains understudied [Tran et al., 2017]. (3) there are plenty of noise peaks blending with the real ions that limits the determination of correct sequences.

To cover the above issues, early pioneering sequencing models integrate graph algorithms and dynamic programming to reduce the task complexity [Dasari et al., 2010]. However, these methods set strong assumptions, and the accuracy of the identification results is restricted. Later on, deep learning technology was introduced to peptide sequencing [Tran et al., 2017; Zhou et al., 2017; Tran et al., 2019]. These methods exploit the encoder-decoder paradigm that firstly utilizes CNN to encoder spectrum and then adopt an RNN-based decoder to generate the output sequence, leading to promising results for this task. Nevertheless, during inference, most sequence models employ a common decoder mechanism using a greedy or beam search. That is, they always predict the next amino acids with top local score and produce the total peptide in one stage. Notably, such a mechanism can miss correct peptides at early steps. In contrast, our hierarchical learning framework incorporates high-confidence guiding tags and a multi-stage sequence extension process to compensate such errors and effectively reduce the search space.

3 Methods

3.1 Task Formulation

Peptide sequencing is the task of automatically producing a peptide sequence ˆP= {ˆa1, ˆa2, . . . ˆaT} to identify a given spectrum S, where ˆat ∈ A is the predicted amino acid letter, A is the space including 20 candidate amino acids, <s> and <e> symbol, and T denotes the peptide sequence length.

Our model builds a hierarchical framework with the same target as those one-stage models, but with the additional intermediate layers between the output layer and the input layer. Specifically, we first train the model via maximizing the loglikelihood of each guiding tag conditioned on the input spectrum and the target peptide sequence P = {a1, a2 . . . aT}, and then optimize the model with peptide-level evaluation metrics. We denote the predicted peptide subsequence of the ith extensive stage decoder as ˆPi , i ∈ {1, . . . ,N}, and N is the number of stages. Notably, we treat stage i = 1 as the guiding tag decoder and ˆP1 = {ˆai, . . . , ˆai+k-1} represents the guiding tag of spectrum S, where k is the length of tags and 1 ≤ k ≤ T. As a result, each intermediate sequence decoder provides the increasingly expanding peptide subsequence, and the prediction of the last decoder is taken as the final peptide sequence.

3.2 SpectrumEncoding

Following [Qiao et al., 2019], we select top n=500 most intense peaks and represent the spectrum as a tuple set {( mz , intensity)}n1 to tackle the accuracy-speed / memory trade off problem. Integrated with spectrum feature extraction matrix, the experimental spectrum S is encoded to the spatial spectrum features V as,

Practically, we employ pre-trained T-Net [Qiao et al., 2019] as our spectrum information encoder, which is designed for the kind of order invariant data.

3.3 Hierarchical Decoding

The overall hierarchical decoding framework consists of one guiding tag decoder and a sequence of Transformer-based extensive decoders that repeatedly extend the prediction of a guiding tag from the preceding decoder. The first stage of our model decoding is a guiding tag decoder which generates high-confidence guiding tags based on the global spectrum features. In the subsequent stages, each stage i ∈ {2, . . . ,N} is a sequence expansion decoder which extends the peptide subsequence based on spectrum features and the outputs of the previous stage. More formally, our proposed hierarchical decoder handle the final peptide sequence as:

In addition, we consider the hidden states of the preceding stage to provide the following stage weighted spectrum regions for better peptide sequence prediction. In the following, we will introduce the adopted guiding tag decoder, tag extension decoder, and refining module in detail.

Guiding Tag Decoder

We start by decoding in a guiding tag search in the first stage (i = 1), where we learn a guiding tag decoder with an Transformer network, named TMtag. At each time step t ∈ [1, k], the input to TMtag consists of the previous output sequence

ˆP1<t = {ˆai, . . . , ˆai+t-1} and the intial spectrum features V. Hence, we get the condition probability:

In practical terms, instead of applying RNN or LSTM, we introduce the basic Transformer model to the peptide sequence generation in this task. In our preliminary experiments, we have found that the Transformer can achieve better performance in this task, and thus we apply this model to the sequence decoder.

Briefly, the Transformer decoder consists of an embedding layer and multiple decoder layers. Each decoder layer has a mask self-attention module and a point-wise feed-foward network (FFN). The detail tag decoder flow is described as:

Sequence Extension Decoder

In the subsequent stages, each ith sequence extension decoder predicts the tth remaining amino acid ˆait based on the spectrum featrues V, the refining weights αi-1 t and the previously generated peptide sequence ˆPi-1 t from the preceding decoder. The probability of sequence can be calculated as:

In fact, each sequence extension decoder consists of an TMex network and a refining module. At each time step t, the input to TMex incudes the refined spectrum feature xi-1 , the previous output amino acid at-1, and the output of the preceding TMex. Therefore, the updating procedure of TMex can be written as:

where hi-1 is the hidden state from preceding Transformer and g(·) is the spatial attention function which feeds refined spectrum features as additional inputs to TMex at each time step to emphasise the effective peak information and ignore the noise peaks. When i=2, it represents the information transmits from the guiding tag decoder to the sequence extension decoder; when i>2, it represents the increasely improved process of the sequence extension decoder itself.

Refining Module

As mentioned above, global spectrum features is adopted for our sequence decoder to generate the amino acids. However, in most real cases, each amino acid is only related to a small region of a spectrum. Directly incorporating the whole spectrum feature for amino acid prediction can lead to sub-optimal identification results due to the noises integrated from the irrelevant regions [Vaswani et al., 2017].

Towards that end, the attention mechanism has been proposed to effectively improve the performance of peptide identification [Xu et al., 2015]. It typically produces a spatial feature map attending spectrum peaks regions relevant to each predicted amino acid. In this work, to extract more available spectrum information for each amino acid prediction, we adopt a Transformered-based cross-modality module to filter out noises gradually and pinpoint the effective peaks that are highly relevant to the current sequence prediction [Anderson et al., 2018]. In each sequence extension stage i, our refining model operates on both spectrum features V and importance weights αi-1 t from the preceding stage.

Formally, for the time step t of stage i, the work process of our refining model can be defined as:

where TMref denotes Transformered-based refining layer and please note that when i = 1, we set α1 t to zero.

The policy network determines a policy pθ1:i which receives a state including preceding subsequence outputs, internal state and spectrum features, and produce an action ˜ait corresponds to the next amino acid at time step t. Once we get the complete predicted peptide sequence ˜Pi , the agent will acquire a reward r(˜Pi ). The goal of RL-based training is to minimize the expected punishments of multi-stage decoding as,

where ˜Pi = {˜ai1 , . . . ˜aiT } and ˜ait is sampled from the stage i at time step t. r(˜Pi ) is calculated by comparing the generated peptide sequence to the corresponding ground-truth sequence. On this basis, we can compute the expected gradient with the Monte-Carlo to gain ˜Pi from pθ1:i as:

4 Experiments

4.1 Experimental Preparation

Datasets. All the experiments are conducted on five public data sets from different labs and species. Table 1 presents the basic information of the data sets where "#Spectra" represents the number of spectra used in training or testing. The forms of these data were all high-resolution HCD. OpenpFind [Chi et al., 2018] was employed to deal with these raw data sets and the five data sets were searched against the corresponding reviewed database of human, mouse, E.coli, and yeast, respectively, which were all downloaded from Uniprot and their versions are consistent with [Chi et al., 2018]. To be specific, we configure the precursor ion tolerance as 20 ppm and the fragment ion tolerance as ±20 ppm. We also controlled the FDR at 1% at the spectrum level. Furthermore, in order to maintain the matching quality of all spectra in these data sets, we removed the PSM whose matched peak number was less than its peptide length as well as the spectra with peptide length longer than 20. Finally, ∼ 920,000 high-quality PSMs were acquired for later experiments. Importantly, the peptide sequences identified from Open-pFind were assigned to the corresponding MS/MS spectra and then used as final ground truth for testing the correctness of sequencing results.

3.4 Training Procedure

The introduced hierarchical framework results in a deep architecture. Correspondingly, it tends to cause the vanishing gradient problem during training, where the magnitude of gradients decreases dramatically when backpropagated through multiple intermediate sequencing layers [Anderson et al., 2018]. An effective approach to tackle this problem is to integrate supervised training objectives for the intermediate sequencing layers. Each stage of the hierarchical subsequence decoder is trained to predict the peptide sequence repeatedly. Here, we first introduce a cross-entropy (XE) loss to optimize the network parameters, i.e.,

where at denotes the ground-truth amino acid letter, and θ1:i is the parameters up to the stage-i peptide sequence decoder. We thus acquire the overall training objective for the full architecture via adding up the losses at each stage i:

where pθ1:i (at|P<t, S) represents the confidence probability of amino acid at given by the i-th decoder. The weights of the models are shared across all time steps.

However, optimizing the loss function LXE is usually not sufficient, since the current log-likelihood training objective causes the discrepancy problem. To be specific, the decoder is trained to focus on the correctness of predicting each amino acid separately. However, at each step in the test stage, the decoder is fed with the predicted amino acid from previous step rather than ground truth. This leads to the gap between training and test and linits the performance in test. To address this gap, we can regard the peptide sequence generation process as a reinforcement learning process. In detail, given an environment, we want to train an agent to take an action (next amino acid) conditioned on the current environment (spectrum features and previously generated amino acids). After producing a complete peptide sequence, the agent will receive a peptidelevel reward and update its internal state accordingly.

Inspired by [Ren et al., 2017; Gu et al., 2018], we fine-tune our multi-stage sequence generation model with RL technique. The sequence decoder of each stage can be considered as an agent that consistly interacts with the environment.

Compared approaches. We compared the following stateof-the-art peptide sequencing methods: PEAKS [Ma et al., 2003], Novor [Jeong et al., 2013] DeepNovo [Tran et al., 2017] and DeepNovoV2 [Qiao et al., 2019]. Note that the first two methods are constructed with a traditional search strategy, while the last two methods are deep learning-based.

Evaluation metrics. In this paper, the generated amino acid can be regarded as correct when the mass difference between the predicted amino acid and a ground-truth amino acid is less than 0.1 Da, and the prefix mass before them as well as the suffix mass behind them are different by less than 0.5 Da. Following [Qiao et al., 2019], we adopt three types of metrics: precision, recall and area under curve (AUC) to evaluate the performance of peptide sequencing. More subdivided, the ratio of the total number of matched amino acids over the total number of amino acids in the generated peptide sequences is considered as amino acid level precision while the fraction of true peptide sequences in total predicted peptide sequences is served as peptide level precision. Similar definitions can be applied to recall and AUC as well.

Implementation details. The structure of our DeepTag model was presented in Figure 1. We utilized T-Net [Qiao et al., 2019] as our spectrum feature extractor. The final output of T-Net was resized to 256 dimensions. The length of the guiding tag was set to 5 (k = 5) and the total number of the stage was set to 3 (N = 3). For guiding tag decoder, we use the small transformer (dmodel = 256, dhidden = 256, pdropout = 0.1, nlayer = 3, and nhead = 2). For extended decoder, we use the base transformer by [Vaswani et al., 2017](dmodel = 512, dhidden = 512, pdropout = 0.1, nlayer = 3 and nhead = 8). We first train our model under the cross-entropy cost using Adam optimizer (lr=0.002) and a momentum parameter of 0.9. Later on, we adopt the proposed RL-based approach on the just trained sequencing model to further optimize. During this stage, we use Adam with a learning rate of 0.0002. After each epoch, we evaluate the model performance on the validation set and choose the sequencing model with the best rewarding. Overall, our model was first trained on the Mann-Human-QE data set, and then tested on Mann-Mouse-QEHF for cross-species validation and Gygi-Human-QE data set for cross-lab validation. The rest of the data sets were adopted to test the robustness of our model. Please note that the training dataset and testing dataset come from different species [Zhou et al., 2017]. The cross-validation is used to guarantee unbiased training and testing and does not give our model any advantage.

4.2 Comparing with State-of-the-art Methods

Table 2 summaries the performance comparisons between the state-of-the-art peptide sequencing models and our proposed DeepTag on different species and labs datasets. In general, our DeepTag consistently exhibits better performance than other sequencing models, which include the traditional search methods (PEAKS, Novor) and deep learning-based methods (DeepNovo and DeepNovoV2). The PR score of our DeepTag can achieve 0.289 on the Mann-Mouse-QEHF dataset, which is to-date the best performance and makes the absolute improvement over the best competitor DeepNovoV2 by 2.3%. The performance improvements generally demonstrate the key advantage of incorporating multi-stage decoding for peptide sequencing. In particular, we can observe from the two types in Table 2 that deep learning-based approaches outperforms top human-designed search approaches ∼ 5.0 % in AAR, ∼ 2.1 % in AAP and ∼ 6.5% in PR. In our opinion, compared with the traditional search methods, the model based on deep learning has greater advantages in extracting spectral information and results in accurate peptide sequence.

On the other hand, we should also be aware that all sequencing tools report confidence scores for their predictions, and setting a higher threshold of confidence score will lead to a smaller part of peptides with high precision but will make the rest of the dataset without results [Tran et al., 2017]. Hence, it is reasonable to depict precision-recall curves and incorporate the area under curve (AUC) as metrics of peptide sequencing quality. Figure 3 and 2 display the AUC of different peptide sequencing methods on different data sets and the precision-recall curves on Mann-Mouse-QEHF dataset respectively. We can view that our multi-stage DeepTag model still maintains superiority against other sequencing methods. For example, for Mann-Mouse-QEHF dataset, the AUC of our DeepTag was 54.8% higher than that of PEAKS ((0.48-0.31) / 0.31 = 0.548) and 37.1% higher than that of Novor ((0.48-0.35) / 0.35 = 0.371). In addition, DeepNovoV2 and our method often came in the first two places, probably because of the improvement of deep learning technology in spectrum information encoder ( both models adopts T-Net to handle spectra ). In summary, the sequence generation method based on deep learning generally present better performance compared with traditional searchbased strategy. More importantly, extensive experimental results demonstrated the improvement of our method was efficient and reliable.

4.3 Neural Network Architecture Analysis

In this paper, we propose a novel hierarchical framework whose architecture is worth further looking into. A natural question arises "why do we choose these structures here?". To answer this question, we construct an ablation study on four variants based on our neural network structure: (1) we implement a one layer Transformer-based guiding tag decoder and one layer Transformer-based sequence extension decoder model named as TM1+1 layers. (2) We add two additional Transformer networks after one layer Transformer sequence extension decoder model, which is named as TM1+3 layers. If the number of layers is not indicated, then N = 3 (1+2) in default. Then, we implement two types of refined-based peptide sequencing models: (1) the output of spectrum features are directly input to the hierarchical decoder, which is named as TM+no Refine. (2) the soft-refined model (TM+RefineSoft) proposed by [Xu et al., 2015].

More concretely, in this experiment, we first train the baselines and our proposed DeepTag with a standard crossentropy loss. We report the performance of our model and the baselines in Table 3. We can find that our hierarchical learning framework achieves the best performances in all metrics. The soft refining models, TM+RefineSoft, provides slightly lower performance than our DeepTag. Note that directly adding extra one additional Transformer layer in TM1+3 layers decreases the performance of our DeepTag as the model may experience overfitting. Our hierarchical approach which optimizes the network gradually with the intermediate supervision can effectively avoid overfitting to some degree. We also view that the refining mechanism (TM+RefineSoft and DeepTag) can significantly improve the performance of peptide identification, compared with TM+no Refine. After optimizing the model with crossentropy loss, we optimize with the RL-based algorithms to fine-tune the models. The performance of five models are presented in Table 4. Similar to the previous observations, our DeepTag still obtains significant gains across all metrics.

5 Conclusion

Peptide sequencing is a challenging problem that incorporates both pattern recognition and optimization in data analysis. In this paper, we propose a hierarchial multi-stage framework that utilized a refining module in conjunction with multiple Transformer networks to improve MS/MS analysis. Our model generates peptide sequences from high-confidence guiding tags to both sides extended sequences, which we found to be very beneficial for peptide identification. The model was compared with other prevalent peptide identification tools and experiments show that our DeepTag achieves higher precision at both amino acid and peptide levels. Besides, the effectiveness of various network architecture was investigated in detail.