Change Matters: Medication Change Prediction with Recurrent Residual Networks

1 Introduction

In recent years, deep learning has demonstrated initial success in potentially assisting clinical decision-making [Almirall et al., 2012; Choi et al., 2017; Xiao et al., 2018; Mao et al., 2019]. Among others, the medication recommendation task has drawn lots of research interest [Wang et al., 2017; Wang et al., 2019; Shang et al., 2019a; Shang et al., 2019b; Zhang et al., 2017; Killian et al., 2019; Wang et al., 2018]. The common strategy of medication recommendation learns representations for medical entities (e.g., diagnoses, medications) from electronic health records, and use the learned representations to predict medications that fit the patient's health condition while avoiding adverse drug interactions.

Many existing works focus on recommending the full set of medications in a visit [Zhang et al., 2017; Shang et al., 2019a;

Shang et al., 2019b; Xiao et al., 2018], which can be quite redundant from previous visits. Because the medications often remain stable over time with large overlaps between consecutive visits. For example, we investigate the Jaccard coefficient over consecutive visits on MIMIC-III data [Johnson et al., 2016] (Fig. 1). Among patients with multiple visits, although most of them are diagnosed with different conditions during consecutive visits (mean Jaccard below 0.2), the sets of medications remain stable (mean Jaccard around 0.5).

However, such medication patterns were rarely explored and leveraged to augment medication recommendation tasks. Challenges mainly arise from (1) how to accurately characterize the changes of patient health condition for each time step, and (2) how to correctly identify the medication changes based on health changes.

To fill in the gap, we propose a new recurrent residual learning approach, named MedicatIon Change pRedictiON (MICRON) to predict medication changes and simultaneously model longitudinal medical history. MICRON is enabled by the following technical contributions.

• Efficient representation of changing health conditions. MICRON uses a residual health representation to sequentially update changes in patient health conditions, which provides more efficient model inference than RNN-based alternatives.
• Explicit change set prediction: MICRON decomposes the medication change prediction task into predicting two sets: (1) the removal set that removes previous medicines that are no longer needed; and (2) the addition set that brings in new medicines for newly developed diseases. Two sets are modeled by a recurrently updated medication vector and addition and removal thresholds selected at the high-confidence

region from validation ROC-curve and thus could provide reliable inclusion-exclusion criterion.

We evaluated MICRON against state-of-the-art models on two real-world patient datasets: one inpatient MIMIC-III data and one outpatient dataset. MICRON outperforms the best baseline GAMENet [Shang et al., 2019b] with 3.5% and 7.8% relative improvement in F1 measure, respectively. In addition, MICRON achieves 1.5× speed-up in training and inference compared with GAMENet. Codes and baselines can be found here 1 . A long version of the paper is also available 2 .

2 RelatedWorks

Rule-based models [Almirall et al., 2012; Chen et al., 2016] typically rely on human-designed clinical guidelines, which require huge efforts from clinicians. For example, [Lakkaraju and Rudin, 2017] optimizes a sequence of if-then-else rules, which maps the patient status into the prescription decision.

Instance-based methods extract patient features only from current visits. [Zhang et al., 2017] formulated the medication recommendation as a multi-instance multi-label (MIML) task and proposed a content-attention mechanism-based sequence-to-sequence model. [Wang et al., 2017] jointly embedded diseases, medicines, patients, and their corresponding relations into a shared space by the knowledge graph, which requires multiple external data sources.

Longitudinal approach [Wang et al., 2018; Xiao et al., 2018; Bhoi et al., 2020; Yang et al., 2021] is a popular approach that captures the sequential dependency in patient treatment history. [Choi et al., 2016; Bajor and Lasko, 2017] modeled the longitudinal patient history by RNNs for various clinical predictive tasks. [Shang et al., 2019b] and [Le et al., 2018] adopted memory-based networks with RNNs to handle the dependency among longitudinal medical codes. A recent work [Yang et al., 2021] exploits molecule structural information to improve safe recommendation.

Compared with existing works, MICRON is based on a new perspective that focuses on predicting the changes. This is more realistic since clinicians usually update patient prescriptions by only a small proportion for a patient's new visit.

3 Method

3.1 ProblemFormulation

Definition 1 (Patient EHRRecords) Patient EHR records are usually represented by an ordered sequence of tuples. For a patient j, we denote his/her clinical documentaries as Xj = [x(1) j , x(2) j , x(3) j , . . . ], where the tth entry, x(t) j , records the information of the tth visit, such as diagnoses, procedures and prescription information. In the paper, x(t) j = [d(t) j , p(t) j ,M(t) j ], where d(t) j ∈ {0, 1}|D| and p(t) j ∈ {0, 1}|P| are multi-hot diagnoses and procedure vectors, while D and P are the overall diagnosis and procedure sets. M(t) j ⊂ M is the tth medication set and M is a set for all possible medicines. We denote the visitwise medication addition (new) and removal (old) sets as N (t) target , O(t) target ⊂ M, separately, which naturally follows the equality, M(t) = (M(t-1) ∪ N (t) target ) \ O(t) target .

1 https://github.com/ycq091044/MICRON

2 https://arxiv.org/abs/2105.01876

Problem 1 (Medication Change Prediction) Medication change prediction aims at determining the medication addition set N (t) and the removal set O(t) at visit t, given the last prescription, ˜M(t-1) and patient health history [d(1) , . . . , d(t) ] and [p(1) , . . . , p(t) ]. The model aims to minimize the gap between current estimation ˜M(t) = ( ˜M(t-1) ∪ N (t) ) \ O(t) and real prescriptions M(t) , and also control the incidence of DDIs as denoted by A ∈ {0, 1}|M|×|M| , where Aij = 1 implies that medicine i and j could interact.

3.2 MICRONMethod

Overview As shown in Fig. 2, MICRON has three modules: (1) a patient representation module that embeds diagnosis and procedure codes into latent health representation; (2) a prescription reconstruction module (training phase), where MICRON trains on consecutive pairs of visits and learns residual medication representations under a new reconstruction design; (3) a medication updating module (inference phase) for model inference, where MICRON initializes with previous medication information. For each subsequent visit, MICRON only requires an update of patient health status and then will predict the changes in the existing medications.

The key difference between MICRON and existing medication recommendation models [Shang et al., 2019b; Choi et al., 2016] is that while these models learn global sequential patterns using RNNs, MICRON learns sequential information locally (by every two consecutive visits) and propagates them visit-by-visit to preserve the longitudinal patient information.

3.3 Patient Representation

Patient representation aims to learn a compact and indicative vector to represent a patient's status. In a clinical visit, doctors will recommend medications based on diagnosis and procedure information. Our module also feeds on these two features. Since MICRON is proposed for generic patients, we leave the subscript notation in the following.

Diagnosis and Procedure Encoders. For the tth visit, the input features, d(t) ∈ R|D| and p(t) ∈ R|P| , can be extracted from clinical documentary. d(t) is the multi-hot diagnosis vector, while p(t) is the procedure vector. Following the similar strategy in [Zhang et al., 2017; Shang et al., 2019b], we transform these two vectors into the embedding space using mapping matrices Ed ∈ Rs×|D| and Ep ∈ Rs×|P| (s is the size of embedding space),

During training, these two tables are shared among all visits and patients. The results, d(t) e and p(t) e , are of the same dimension Rs .

Patient Hidden Representation. To achieve one compact health representation, d(t) e and p(t) e are further concatenated and parametrized by a health representation network, NEThealth, denoted as residual health representation, which encodes the changes in clinical health measurements, indicating an update in patient's health condition. Naturally, the health update r(t) will cause an update in the resulting medication representation u(t) . Our motivation is that if NETmed can map a complete health representation (e.g., h(t) ) into a complete medication representation (e.g., ˆm(t) ), then a residual health representation should also be mapped into an update in the same representation space through NETmed. In other words, r(t) and u(t) shall also follow the same mapping function, NETmed,

To learn Eqn. (3) and (4), we could use the medication combinations in the dataset as supervision, however, it is hard to formulate a direct supervision for Eqn. (5). A simple idea is to model the addition and the removal medication sets separately (as we show in the experiment that separate modeling DualNN does not work well). Therefore, we consider reconstructing u(t) from ˆm(t-1) and ˆm(t) by both unsupervised and supervised regularization.

Unsupervised Residual Reconstruction. To model the medication changes, we design a reconstruction loss. For Eqn. (3), (4) and (5), the inputs follow a residual relation: h(t-1) + r(t) = h(t) . Naturally, we also impose a similar relation in the medication output layer by introducing an unsupervised reconstruction loss (σ(·) is a Sigmoid function),

which is calculated with vector 2-norm. This reconstruction loss enforces the reconstructed recommendations from ˆm(t-1) and the residual u(t) to be close to the recommendations given by ˆm(t) . We show in the experiment that L(t) rec is essential for learning the residual.

which outputs an integrated health representation h(t) ∈ Rs . In this paper, we use affine function (one layer neural network without activation) for NEThealth.

Starting from h(t) , the model architecture differs in training and inference. Next, we elaborate on these two phases.

3.4 Training: Prescription ReconstructionModule

Our MICRON trains on two consecutive visits, e.g., the (t-1)th and the tth visits. Given the health representations, i.e., h(t-1) and h(t) , a straightforward way for recommending medications is to learn a mapping, i.e., a prescription network, NETmed : Rs ↦→R|M| , from hidden embedding space to medication space for two visits, separately.

where ˆm(t-1) , ˆm(t) ∈ R|M| are medication representations and each entry quantifies a real value for the corresponding medicine. In the paper, NETmed is implemented as a fully connected neural network. To obtain the actual recommendations, a trivial way [Shang et al., 2019a; Shang et al., 2019b] is to apply a medication output layer, which consists of a Sigmoid function σ(·), followed by a predefined threshold δ, picking up medicines with larger activation value. However, in this paper, we hope to utilize and emphasize the dependency usage between h(t-1) and h(t) in the model.

Residual Medication Representation. Formally, the difference between h(t-1) and h(t) , i.e., r(t) = h(t) - h(t-1) , is Supervised Multi-label Classification. To jointly modeling a low DDI output, we introduce three differentiable loss functions to improve ˆm(t-1) and ˆm(t) , so as to achieve a better reconstruction u(t) .

• Drug-Drug Interaction Loss. Since adverse drug-drug interaction (DDI) is a leading cause of morbidity and mortality in clinical treatments [Percha and Altman, 2013], we penalize the presence of DDIs in the output medication representation, ˆm(t) . First, we transform it by Sigmoid function, ˆo(t) = σ( ˆm(t) ), and then design the DDI loss as,

where A is the binary DDI matrix, extracted externally [Tatonetti et al., 2012] and Aij indicates that medicine i and j have interaction or not. The term Aij · ˆo(t) i · ˆo(t) j is the a scalar product, which is the interaction penalty for medicine i and j, and ˆo(t) i is the i-th element of the vector. Since we care about the DDI rate in the reconstructed representation, this loss only applies to the current visit t.

• Binary Cross-entropy Loss. In addition, we also extract real medication set as supervision. Assume a multi-hot vector m(t) ∈ {0, 1}|M| is the vectorization of the target medication set M(t) . We adopt binary cross entropy (BCE) loss,

where subscript i indicates each element of the vectors. For this loss function, we compute on both ˆm(t-1) and ˆm(t) .

• Multi-Label Margin Loss. Then, we employ margin-based loss to enlarge the gap between the recommended medications and the unselected ones. Since ˆo(t) i ∈ (0, 1), the margin is set 1 in our paper.

We also consider to penalize both of the visits, i.e., calculating L(t-1) multi and L(t) multi using this loss.

These three losses use external supervision to optimize the prescription network, so that during inference, our MICRON would predict medication changes more accurately.

Overall Loss Function. In the training process, we hope to find optimal values for embedding tables, Ed and Ep, parameter matrices in NEThealth and NETmed. The loss functions are combined by weighted sum,

where λi, i = 1, 2, 3, 4, are different weights for four types of loss functions, and γ is introduced to balance two consecutive visits. During the training, one batch contains all visits of one patient, and the loss is back-propagated after each batch. In the paper, we treat the weights as hyperparameters. In Appendix, we also prototype a momentum-based method to select the weights automatically.

3.5 Inference: Medication UpdatingModule

To predict medication changes, it is essential to maintain a medication combination. We hope that for the subsequent visit, it would be enough to derive an update in the combination based on new diagnosis or procedure information. However, like Risperdal for treating schizophrenia and bipolar disorder, some medicines will not be prescribed based on only one or two visits, and it might need long-term clinical observation. We therefore also maintain a medication vector, where each element quantifies the cumulative effect of a medicine. After clinical visits, each element in the vector will increase or decrease based on the updates of patient health status. Essentially, the medication vector is like the memory cell in RNNs, which is refreshed visit-by-visit. Once it is above or below certain thresholds, the medicine will be added or removed from the current sets. More concretely, the medicine change prediction follows three steps.

Step 1: Medication Vector Update. Specifically, for the tth visit of a patient, the medication changes start from a medication vector, ˜m(t-1) ∈ R|M| , and a medication set, ˜M(t-1) ⊂M. The model first updates the vector based on a residual health representation, r(t) ,

where r(t) is calculated by h(t) -h(t-1) (defined in Eqn. (2), NEThealth), which is implemented as an affine function. We use an efficient smart inference module to calculate r(t) , in case only the updates in medical codes (e.g., diagnosis and procedure) are accessible. We specify it in Appendix.

Step 2: Addition and Removal. Then, based on the updated medication vector, ˜m(t) ∈ R|M| , we identify which medicines are ready to add or remove. We design two thresholds (δ1 is for the addition set, while δ2 is for the removal set, where 1 ≥ δ1 ≥ δ2 ≥ 0) to control the size of changes. Specifically, we first apply a Sigmoid function σ(·), and then the addition and removal sets are generated by applying the thresholds δ1 and δ2 element-wise,

where N (t) (O(t) ) is for addition (removal) set, and subscript i enumerates the index of ˜m(t) . Note that, N (t) ∩ O(t) = ∅. For two thresholds, if δ1 = 1 and δ2 = 0, then N (t) = O(t) = ∅; in another case, δ1 = δ2, then "medication change prediction" becomes "full medication prediction".

The thresholds δ1 and δ2 are selected based on the receiver operating characteristic (ROC) of each medicine. Specifically, we load the pre-trained MICRON on the validation set. For each medicine, we collect cut-off thresholds of the ROC curve in descending order. δ1 is the average of 5-percentile over all medications, while δ2 is based on 95-percentile. Essentially, δ1 will provide a low false negative (FN) rate, while δ2 ensures a low false positive (FP) rate.

Step 3: Medication Set Update. Next, we apply the changes (addition and removal) in the existing medication set. The generation of a new combination is given by set operations,

where we use set union and subtraction operation. N (t) and ˜M(t-1) could have overlaps, while O(t) could also contain medications that are not in ˜M(t-1) . The overlaps will not affect the final recommendation results due to set operations.

To sum up, the model begins with a medication vector, ˜m(t-1) , and a medication set, ˜M(t-1) , which are provided by the previous visit. During the current visit, MICRON uses the update of patient status as input and walks through the above three steps to finish one round of medication change, as well as to update ˜m(t) and ˜M(t) for the next visit.

4 Experiments

We evaluate MICRON against several baselines in both inpatient and outpatient datasets. We focus on answering the following questions:

• How does MICRON perform against the baselines in medication and change prediction?
• How does MICRON perform in model efficiency?
• How do different components in MICRON contribute to accurate recommendations?

4.1 Experimental Setup

Dataset. We consider a benchmark inpatient dataset: MIMIC-III [Johnson et al., 2016], and a private outpatient dataset: IQVIA PharMetrics Plus (see processed statistics in Table 1). Details of dataset descriptions, preprocessing, hyperparameter selections can be found in Appendix.

Baselines. We consider the following baselines (SimNN and DualNN are designed by ourselves).

• SimNN use the same patient representation h(t) as MICRON and then learns a simple 3-way classifier for each medicine (add, remove, and remain) with the cross-entropy loss.
• DualNN also starts from patient representation h(t) and then diverges to two different neural networks. The first one is for addition and the second for removal. Each neural network classifier uses the binary cross-entropy loss.
• LEAP [Zhang et al., 2017] is an instance-based approach that uses a sequence to sequence model with reinforcement aftermath fine-tuning. This method generates a list of medications based on the diagnoses in the same visit.
• RETAIN [Choi et al., 2016] is a longitudinal predictive model, which designs a specialized attention model over RNN. It learns the temporal dependencies between clinical visits and makes medication recommendations.
• GAMENet [Shang et al., 2019b] is also a longitudinal model, which uses RNN, memory network, and graph neural network. It predicts medications using historical prescriptions as reference.

Evaluation Strategy and Metrics. We use evaluation metrics such as DDI rate, Jaccard Similarity, F1-score similar to evaluate the overall recommended medications, as other related works [Shang et al., 2019b; Zhang et al., 2017]. Also, we design new error metrics to evaluate the accuracy of medication changes: Err(add) and Err(remove).

Err(add) computes the sum of false positive part and false negative part (white region in Fig. 3) between the predicted addition set N (t) and the target addition set N (t) target . The target addition set is calculated by

where M(t) is the target medication set in the tth visit, and ˜M(t-1) is the predicted medication set at the (t - 1)th visit. For a particular patient j, this metric is calculated from the second visit, where the medication changes start,

where A\B is the set subtraction, and V (j) means total number of visits of patient j. Finally, we average over all patients and get Err(add). Similarly, we design Err(remove) to denote errors for removal. Also, we report model size and training/inference time as another two measures.

Since the evaluation is on medication change prediction, we assume the earliest visit appeared in the window as the "first" visit of a patient j, where we could extract the initial medication set, ˜M(1) j = M(1) j , and the initial medication vector, ˜m(1) j = ˆm(1) j . For a fair comparison, the evaluation of all models starts from the "second" visit. The definition of other metrics can be found in Appendix.

4.2 Experimental Results

We conduct experimental comparison based on five different random seeds and show the mean metric values in Table 2. Due to space limitation, the standard deviation results are reported in the Appendix. The result of MIMIC-III and IQVIA are reported together and separated by "/". The last two metrics, Model Size and Train(epoch), are based on MIMIC-III dataset. For the other five metrics, we select the best results in bond font and use underscore to select the best baseline. We also report the improvement ∆ of our MICRON over the best baseline. For DDI, Err(add), Err(remove), the lower the better, while for Jaccard and F1, the higher the better.

MICRON outperforms most baselines in both inpatient and outpatient settings, especially for Jaccard and F1 metrics. LEAP gives a relatively good DDI measure on two datasets; however, its performance is weaker than other baselines in terms of accuracy. Although SimNN, DualNN, and RETAIN are implemented from very different perspectives, the former two are instance-based while the latter uses sequence modeling. They show neck-to-neck performance on MIMIC-III. For outpatient medication change prediction (on IQVIA), RETAIN shows strong performance while some recent state of the art baselines failed, such as GAMENet. We hypothesize that time spans between two visits can be much longer for outpatients, and thus the stored memory can be less trustworthy in GAMENet. By learning an effective residual representation, MICRON provides more accurate and safe medication recommendations for inpatient or outpatient settings. Also, MICRON requires much fewer parameters than the state-ofthe-art approaches, which is more efficient.

We also test the model's stability and do a T-hypothesis testing of MICRON on each metric. As a summary, most of the p-values are less than 0.001 (mean p-value at 6.2e-5), except in two cases on IQVIA: the DDI rate compared to LEAP and the Err(remove) compared to RETAIN.

4.3 Ablation Study onModel Components

In this section, we verify the effectiveness of different components in MICRON. Specifically, we conduct ablation studies on MIMIC-III and test on the following variants:

• (i) MICRON w/o Lrec. We remove the unsupervised loss during training and solely trained on the supervised loss.
• (ii) MICRON w/o ˜m(t) . We do not maintain the medication vector, ˜m(t) , and only utilizes the update feature information, r(t) , between two visits;
• (iii) MICRON w/o Lmulti. We remove Lmulti, and it will be less confident to use thresholds, δ1 and δ2;
• (iv) MICRON w/o Lddi. We remove DDI loss, and the model probably would provide high-DDI combinations;
• (v) MICRON w/o δ1, δ2. We set δ1 = δ2 = δ, which implies medications with score above or equal δ being added, and medications with score less than δ being removed. This is a common strategy used in previous works: δ = 0.5 in [Shang et al., 2019b] and δ = 0.3 in [Shang et al., 2019a] (this model requires ontology information, so it is not included as baseline). We use δ = 0.5 for this model variant.

The comparison results with variances (after ±) are shown in Table 3. Overall, all other variations perform better than variant (i) and (ii), highlighting that the reconstruction design and the medication vector are essential in the model. Without medication vector ˜m(t) , the model cannot retain the longitudinal information, thus variant (ii) provides poor results. We also notice that without DDI loss, variant (iv) outputs a significantly higher DDI rate, and MICRON shows slightly better results than model variant (iii) without Lmulti. By integrating all components, MICRON achieves a more balanced and stable performance in all metrics.

5 Conclusion

This paper tackles the medication change prediction problem and proposes a recurrent residual learning model, named MICRON, for predicting medication changes. We compare our model with state of the art approaches and show its effectiveness and efficiency on inpatient MIMIC-III dataset and a proprietary outpatient IQVIA dataset. This paper uses the existing prescriptions as a gold standard. The efficacy of the recommendation is evaluated by comparing it with the prescriptions given by the dataset, which might be a limitation. In the future, we consider performing clinical user studies to evaluate our results.

Acknowledgements

This work was in part supported by the National Science Foundation award SCH-2014438, PPoSS 2028839, IIS1838042, the National Institute of Health award NIH R01 1R01NS107291-01.