--- title: "DO-CoLM: Dynamic 3D Conformation Relationships Capture with Self-Adaptive Ordering Molecular Relational Modeling in Language Models" authors: - "Zhuo Chen" - "Jiahui Zhang" - "Sihan Wang" - "Hongxin Xiang" - "Jianmin Wang" - "Wenjie Du" - "Yang Wang" doi: "10.24963/ijcai.2025/546" url: "https://ijcai.org/proceedings/2025/546" pdf: "https://ijcai.org/proceedings/2025/0546.pdf" venue: "Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence" year: 2025 pages: "4905-4913" track: "Main Track" keywords: - "Natural Language Processing: NLP: Language generation" - "Machine Learning: ML: Multi-modal learning" --- # DO-CoLM: Dynamic 3D Conformation Relationships Capture with Self-Adaptive Ordering Molecular Relational Modeling in Language Models Zhuo Chen, Jiahui Zhang, Sihan Wang, Hongxin Xiang, Jianmin Wang, Wenjie Du, Yang Wang *Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence*, pages 4905-4913. DOI: [10.24963/ijcai.2025/546](https://doi.org/10.24963/ijcai.2025/546) ## Abstract Molecular Relational Learning (MRL) aims to understand interactions between molecular pairs, playing a critical role in advancing biochemical research. Recently, Large Language Models (LLMs), with their extensive knowledge bases and advanced reasoning capabilities, have emerged as powerful tools for MRL. However, existing LLMs, which primarily rely on SMILES strings and molecular graphs, face two major challenges. They struggle to capture molecular stereochemistry and dynamics, as molecules possess multiple 3D conformations with varying reactivity and dynamic transformation relationships that are essential for accurately predicting molecular interactions but cannot be effectively represented by 1D SMILES or 2D molecular graphs. Additionally, these models do not consider the autoregressive nature of LLMs, overlooking the impact of input order on model performance. To address these issues, we propose DO-CoLM: a Dynamic relationship capture and self-adaptive Ordering 3D molecular Conformation LM for MRL. By introducing modules to dynamically model intra-molecular and inter-molecular conformational relationships and adaptively adjust the molecular modality input order, DO-CoLM achieves superior performance, as demonstrated by experimental results on 12 cross-domain datasets. ## Full text ## 1 Introduction Molecular Relationship Learning (MRL) aims to understand interactions between molecular pairs, a critical topic with applications in drug discovery and materials science. For example, drug-drug interactions (DDIs) are vital in pharmacology and drug development, while solute-solvent interactions (SSIs) play a key role in solution chemistry and chemical process design. However, validating these interactions experimentally is time-intensive and costly. Large Language Models (LLMs), with their vast knowledge base and advanced reasoning capabilities, offer a promising solution [Taylor et al., 2022a]. Models like ReactionT5 [Sagawa and Kojima, 2023] and MolTC [Fang et al., 2024] extend LLMs to MRL by leveraging multimodal data, including molecular graphs (2D), chemical properties, and SMILES (1D), for accurate interaction predictions. Despite their potential, these models face unexplored challenges: Figure 1: A molecule can have multiple conformations with mutual interconversion relationships [Skjærven et al., 2011] and the actual molecular interactions are determined by the specific conformations [Mortier et al., 2015]. ![Image](https://ijcai.org/proceedings/2025/fulltext/546/image_000000_20ed73a539c69b1fd9994fb2e528317523508498b59ffe8a045b4a84006218f2.png) Lack of the molecular stereochemistry and dynamics. Existing LLMs primarily rely on SMILES strings and molecular graphs for predicting molecular interactions [Fang et al., 2024; Sagawa and Kojima, 2023], but these 1D and 2D representations fail to capture the 3D information of molecules [Liu et al., 2023] and the dynamic nature of molecular conformations during chemical reactions [Morris et al., 2019]. Essential properties such as stereochemistry (including chirality and stereoisomerism) and molecular dynamics [Andrade et al., 2009] are crucial for understanding molecular interactions and predicting reaction mechanisms [Alonso et al., 2006]. On the one hand, as shown in Figure 1, a molecule has multiple conformations that continuously interconvert [Skjærven et al., 2011]; on the other hand, the interaction between two molecules occurs through the interaction of their specific conformations, determined by their 3D structures [Mortier et al., 2015], such as the binding of small-molecule and target proteins [Wu et al., 2022]. Figure 2: Swapping the molecular input order leads to different predictions and MolTC's results for DDI tasks (Normal) compared to when the SMILES order is swapped (Reverse). ![Image](https://ijcai.org/proceedings/2025/fulltext/546/image_000001_c54de4eb90b91600966e3e8ce38186626fd4a43ca50967f1f7cf7ff026e54508.png) Ignorance of the influence of input order. The performance of LLM is naturally influenced by the input sequence [Berglund et al., 2023], which can have catastrophic consequences in certain MRL tasks. For instance, in DDI tasks, the SMILES order of input molecules should exhibit symmetry, meaning that swapping the SMILES input order should not affect the model's prediction results. However, as shown in Figure 2, swapping the input SMILES order causes the model to produce different predictions, leading to a decrease in MolTC's performance, with accuracy dropping by 11.58% on the Drugbank dataset, 9.24% on the ZhangDDI dataset, 12.81% on the ChChMiner dataset, and 9.08% on the DeepDDI dataset. Recent studies have highlighted this issue [Berglund et al., 2023], which is attributed to the autoregressive training paradigm typically employed by LLMs, leading the model to treat input order as a feature. This can be helpful in some tasks, such as SSI tasks, but it should not affect model performance in symmetric tasks. Notably, in addition to the overall SMILES order, the internal order of molecular tokens also could affect the model's performance. Considering the above issues, we propose DO-CoLM: a Dynamic relationship capture and self-adaptive Ordering 3D molecular Conformation LM for MRL. Specifically, we design the Dynamic Conformational Relationship Capture Module (DCRCM), to address the lack of molecular stereochemistry and dynamics. This module models both intramolecular conformation transformation and intermolecular conformation interaction relationships using heterogeneous graphs based on conformational information. Additionally, we introduce the Adaptive Order Adjustment Module (AOAM), which ensures the model's robustness to perturbations in symmetric input orders and allows it to adaptively find the optimal sequence for asymmetric input orders from the model's perspective. The main contributions of this paper could be summarized as follows: - Given the current limitations of LLM in adequately considering molecular 3D information and dynamics, and the ne- - glect of input order in MRL tasks, we propose DO-CoLM, a multimodal framework that adaptively integrates conformational information and accommodates molecular order. - DO-CoLM enables dynamic learning of relationships between molecular conformations by constructing a heterogeneous graph and adaptively selecting the optimal conformation order, thereby achieving comprehensive molecular information capture and tolerance to molecular encoder tokens variations. - The superiority of DO-CoLM is empirically validated through extensive experiments across 12 datasets from various domains, including DDI, CSI, and SSI tasks. These results demonstrate that our approach significantly outperforms existing methods. ## 2 RelatedWork ## 2.1 Molecular Relational Learning Early research primarily relied on graph neural networks (GNNs) to construct predictive frameworks for MRL [Du et al., ; Fu et al., 2020; Sizhe Liu et al., 2024]. For instance, Nyamabo et al. proposed a substructure-substructure interaction framework [Zhong et al., 2024], which utilizes graph attention network (GAT) layers for substructure extraction and a co-attention layer to model interactions between different substructures. To further capture molecular interactions, Lee et al. [Lee et al., 2023] introduced the Conditional Graph Information Bottleneck (CGIB) model, inspired by the information bottleneck theory. However, their models lack prior knowledge and fail to leverage the advantages of LLMs. ## 2.2 LLMs in the Molecular Domain LLMs have been widely applied in the molecular domain [Zheng et al., 2025; Jablonka et al., 2024]. In 1D, model like MolT5 [Edwards et al., 2022] tokenizes SMILES strings; in 2D, methods such as Text2Mol [Edwards et al., 2021], MolCA [Liu et al., 2023], and DrugChat [Liang et al., 2023] combine molecular graphs with LLMs; and in 3D, MolLM [Tang et al., 2024] and 3D-MoLM [Li et al., 2024] capture spatial features using attention and 3D encoders. Multimodal models like ReactionT5 [Sagawa and Kojima, 2023] and MolTC [Fang et al., 2024] integrate 1D, 2D, and chemical property data for prediction. However, most existing work focuses on single molecules or 1D and 2D data, overlooking the role of 3D conformations in MRL. ## 2.3 Input Order Previous studies show that input sequence order significantly affects model performance. For instance, the improved Seq2Seq model [Vinyals et al., 2015] reveals that even unordered data can benefit from an optimal input order. PointNet [Qi et al., 2017] mitigates order sensitivity using symmetry functions. Work on the reversal curse [Berglund et al., 2023] and inter-modal order [Tan et al., 2024] further highlights order's impact in LMs. However, most focus on single-modal inputs or coarse inter-modal order, neglecting fine-grained token-level order across modalities. Figure 3: The architecture of DO-CoLM. (a) is to encode 3D molecular conformations and dynamically model their relationships. (b) is designed to adaptively adjust the input order and perform modality alignment to generate the output. ![Image](https://ijcai.org/proceedings/2025/fulltext/546/image_000002_7280b322915095cbbee2721945da15632228621fa166d091c19178c386d104f7.png) ## 3 Our ProposedModel: DO-CoLM In this section, we present the architecture of DO-CoLM. As shown in Figure 3, DO-CoLM consists of five key steps. First, the 3D Encoder is used to obtain molecular conformer feature embeddings. Next, the DCRCM dynamically models the intramolecular and intermolecular conformational relationships. Then, the AOAM adaptively adjusts the input order based on the asymmetry of the input data to optimize the model's performance. Afterward, the Alignment module ensures seamless integration by aligning molecular modality data with textual data. Finally, the LLM generates the model's output based on the processed information. ## 3.1 Conformer Encoder Let C a = [C a 1 , . . . , C ana ] and C b = [C b 1 , . . . , C bnb ] represent the conformer pairs for molecule a and molecule b, where C a i and C b i are the i-th conformations, na and nb are the number of conformations for each molecule. We leverage UniMol [Zhou et al., 2023], a powerful 3D molecular feature encoder f, along with attention pooling p, to capture the embeddings of conformer pairs. where Va and Vb represent the sets of conformation feature embeddings for the two molecules. ## 3.2 Dynamic Conformational Relationship Capture Module Due to the dynamic nature of molecular conformations, accurately modeling the transitions and interactions of molecular conformations requires complex simulations and experiments. Therefore, a conformation transition relationship graph based on predefined structures is not feasible. To address this, we propose the DCRCM, which allows the model to dynamically model intramolecular transitions and intermolecular interactions. We define this relationship as the Heterogeneous Molecular Conformation Interaction Graph (HMCG). The HMCG is represented by GHMCG = (V, E,A, T ,R), where V = Va ∪Vb is the set of conformer nodes, E is the edge set with three types: intra-molecular edges within a, intra-molecular edges within b, and inter-molecular edges between a and b. A is the adjacency matrix representing node connectivity, T denotes node types, and R denotes edge types. To model the intra-molecular conformation transformation edges Eintra, we first obtain Aintra through the following algorithm, taking molecule a as an example. where Ua1 represents the representation learning for the intramolecular conformation as the source node, and Ua2 as the target node, θ1 and θ2 represent the learnable vectors for learning the source and target nodes, respectively, and σ1 and σ2 are the activation functions. Such an operation ensures that the generated graph is a directed and asymmetric graph. To reduce the complexity of the graph, we perform a sparsification operation on it. where argtopk(·) returns the indices of the most likely conformation nodes for transformation. The weights of other edges are set to 0. For the conformation transformation edges in molecule b, we adopt the same approach to obtain Ab intra . Since the inter-molecular conformation interaction is symmetric, we use a bidirectional graph to construct the intermolecular conformation interaction edges Einter. The algorithm to obtain the adjacency matrix Ainter representing the edges is as follows: where Ua and Ub represent the relationship representations of the source node and target node for the intermolecular conformation interaction, respectively. Here, we use molecule a as the source node and molecule b as the target node, and Θa and Θb are learnable vectors for learning the relationship between the source node and the target node. After applying the same sparsification operation as in Equation (3), we obtain Ainter. After obtaining GHMCG through the above operation, we aggregate the node information on the graph using Attention(·) and Message(·). The specific computation formula is as follows: Attention(u, e, v) where N(v) represents the set of nodes connected to node v, F τ(u) denotes a fully connected layer with parameters {Wτ(v) , bτ(v) }, ρ(e) represents the type of edge, ∥ represents the concatenation operation, and nh denotes the number of attention heads. Then update the node information. where ⊕ represents the aggregation operation, h(l) v is an aggregated feature representation of the conformation. ## 3.3 Adaptive Order Adjustment Module Due to the sensitivity of large models to input order, different input orders can significantly impact model performance. In our model, in addition to the overall SMILES order, there is also the internal order of nodes which also impacts the performance of the model. For these two types of order, we have designed different strategies to enable the model to achieve optimal performance. The overall order of SMILES. In the DDI task, there are only two possible input molecular sequences: molecular a followed by molecular b, and molecular b followed by molecular a. These two sequences are symmetric and should not affect the model's experimental results. Therefore, we can use a simple operation to enable the model to understand these two different sequences, ensuring that the overall order of the input molecules does not influence the model's performance. This can be achieved by randomly shuffling the input sequence, allowing the model to focus solely on the molecular information itself without considering the positional information it carries. The internal order of nodes. GHMCG comprises n conformer nodes in total, corresponding to n! possible sequences. Achieving order invariance by enabling the model to simultaneously process all these sequences is nearly impossible. Given that different sequences impact model performance to varying degrees, we propose that an optimal sequence exists from the model's perspective, one that cannot be captured through prior knowledge-based ordering. To address this, we hypothesize that each conformer has an inherent priority and allow the model to autonomously determine these priorities and dynamically adjust them based on different situations, thereby uncovering the optimal sequence in the model's view. We use a Learnable Sorting Network K(·) with a learnable parameter for each conformation to allow the model to prioritize the node hi. Then we obtain the sorted order Sb while the initial order is Sa. where n is the number of the conformers. Due to the nondifferentiability of argsort, we employed the Sinkhorn Operator [Cuturi, 2013] to dynamically update the sorting process. Initially, we obtain the transport matrix Q. where Sia and Sj b represent the position values of conformers i and j, ϵ is a control factor. In the iterations, Q is updated as: where K is the size of Q, which means the number of rows and columns, u = 1K. The iterations stop when ∆(vQ⊤u, 1K/K) < η. The max iteration number is depend on ϵ: typically, the smaller ϵ, the larger the number is needed to ensure that vQ⊤u is close to 1K/K. Then we obtain the conformer representation Z by normalizing Q and multiplying with the conformation H. where diag(·) means diagonal operation. ## 3.4 Alignment After acquiring the multi-conformer representations Za and Zb of molecule a and molecule b, the next step is to map them into the backbone LLM's hidden space using projectors fproj1, fproj2, where fproj1 and fproj2 share the same parameters. These projectors act as critical connectors, translating Za and Zb into LLM-comprehensible encodings Ma and Mb. We instantiate fproj1 and fproj2 using trainable projection matrices, which share the same dimensionality as the word embedding space in the language model. More formally, the encodings can be expressed as: where d denotes the feature dimensionality of the LLM and mi represents the embedded representation of a single aligned conformer. ## 3.5 Backbone LLM DO-CoLM uses Galactica1.3B [Taylor et al., 2022b], a decoder-only transformer based on the OPT framework, trained on scientific literature, as its backbone. It excels in interpreting molecular sequences. DO-CoLM leverages Galactica's reasoning to analyze interactions between two molecular sets and their information tokens. The prompt sequence X is as follows: where n denotes the total integrated input length, P represents the task-specific prompt, and P refers to a collection of manually designed prompts, each specifically crafted for the molecular interaction task r. The details of the prompt will be discussed in Section 3.6. The generation process employs a causal mask to produce a response that encapsulates the key interactive properties, with a length of L: The training objective is to predict the target response from the input prompt X. Specifically, the output for the i-th token, represented as ˆxi, is determined based on its preceding tokens as follows for t ∈ (1, L): ## 3.6 Prompt For different tasks related to molecular interactions, we have designed distinct prompts. Here, SMILES1 and SMILES2 represents the SMILES notation, ConEmb1 and ConEmb2 represents multiple conformer tokens Ma and Mb. For DDI tasks, the goal is to enable the LLM to leverage the conformational information of the two input molecules. By doing so, the LLM actively determines and provides an answer as to whether there is an interaction between the two molecules, drawing on their respective conformations to make judgment. ## Prompt for DDI Tasks | Input Prompt | , the information about its conformers is . , the information about its conformers is . Do they have any side effects? | |-----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------| | Target Response | Considering the conformer informa- tion of molecule1 and the conformer information of molecule2, there is a side effect between molecule A and molecule B. | For SSI tasks, where LLMs are less adept, we design a prompt that encourages the model to consider molecular conformations during inference. Instead of a direct answer, the model predicts a range of possible interaction outcomes based on the conformations. It then synthesizes the data to estimate a specific value within that range. ## Prompt for SSI Tasks | Input Prompt | , the information about its conformers is . , the information about its conformers is . What is the solvation Gibbs free en- ergy of this pair of molecules? | |-----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Target Response | Considering the conformation of molecule1 and molecule2, the sol- vation Gibbs free energy of these two molecules is above 2.5 and be- low 3.0, so the accurate value is 2.768. | ## 4 Eeperimental Results ## 4.1 Experimental Setting We evaluate DO-CoLM on well-established downstream molecule interaction tasks involving qualitative and quantitative analysis. Here we provide a overview of our experimental setup. Detailed descriptions are presented in the Appendix C. Datasets. We employ 12 datasets across various domains such as DDI, SSI, and CSI. Specifically, we collect Drugbank (Version 5.0.3), ZhangDDI, ChChMiner, DeepDDI, TWOSIDES, Chromophore, MNSol, CompSol, Abraham, CombiSolv, FreeSolv, and CombiSolv-QM. Here, we process each molecule using RDKit to obtaining several conformations for each molecule. The hyperparameter experiment on the number of different conformations is shown in the Appendix E. Here, 10 conformations are used in subsequent experiments. Baselines. For a comprehensive evaluation, we conduct various baseline methods encompassing distinct categories such as methods based on: GNNs, DL models other than GNN, and LLMs. Specifically, for DDI task, we employ IGIB-ISE [Zhang et al., 2025], MHCADDI [Deac et al., 2019], DeepDDI [Ryu et al., 2018], SSI-DDI, CGIB, CMRL, MDF-SADDI [Lin et al., 2022], DSN-DDI [Li et al., 2023], MolTC [Fang et al., 2024] as the baseline. For SSI and CSI tasks, we utilize D-MPNN [Vermeire and Green, 2021], SolvBert [Yu et al., 2023], SMD [Meng et al., 2023], CGIB, MMGNN [Du et al., 2024], GEM [Fang et al., 2022], GOVER [Rong et al., 2020], Uni-Mol [Zhou et al., 2023] as the baseline. Furthermore, all downstream tasks adopt LLM-based methods, such as Galactica, Chem T5 [Christofidellis et al., 2023], MolT5, MolCA [Liu et al., 2023] and MolTC as the baseline. ## 4.2 EvaluationMetrics We employ prediction Accuracy (%) and AUC-ROC (Area Under the Receiver Operating Characteristic curve) as comparative metrics, while for quantitative tasks, MAE (Mean Absolute Error) and RMSE (Root Mean Square Error) are utilized as the metrics. ## 4.3 Experimental Results and Analysis Due to the limitations of certain models on some datasets, we only present a subset of the results here. For detailed results on other datasets, please refer to Appendix D. Table 1: Comparative performance of various methods in qualitative interactive tasks. The best-performing methods are highlighted in bold, while the second-best methods are marked with * for emphasis. | Baseline | Baseline - Model | Drugbank - Accuracy | Drugbank - AUC-ROC | ZhangDDI - Accuracy | ZhangDDI - AUC-ROC | ChChMiner - Accuracy | ChChMiner - AUC-ROC | DeepDDI - Accuracy | DeepDDI - AUC-ROC | |----------------|-------------------------------------|---------------------------------------------------------|---------------------------------------------------------|---------------------------------------------------------|---------------------------------------------------------|---------------------------------------------------------|---------------------------------------------------------|---------------------------------------------------------|---------------------------------------------------------| | GNN Based | IGIB-ISE SSI-DDI DSN-DDI CMRL CGIB | 95.16±0.37 94.12±0.32 94.95±0.12 94.93±0.12 94.67±0.34 | 98.83±0.56 98.28±0.29 99.01∗±0.11 98.76±0.11 98.53±0.24 | 88.80±0.35 87.02±0.32 87.55±0.11 87.68±0.32 87.82±0.73 | 94.75±0.41 93.66±0.38 94.83±0.28 94.58±0.21 93.98±0.61 | 94.97±0.38 93.16±0.21 84.20±0.18 94.25±0.26 94.35±0.36 | 97.84±0.32 97.91±0.12 94.05±0.26 98.34±0.12 98.45∗±0.32 | 96.24±0.35 94.97±0.23 95.44±0.28 96.27±0.34 96.13±0.49 | 98.52±0.31 98.41±0.30 98.02±0.15 98.97∗±0.29 97.98±0.64 | | ML Based | DeepDDI MHCADDI MDF-SA-DDI | 93.05±0.25 79.30±0.80 93.96±0.32 | 98.36±0.54 86.23±0.45 97.59±0.29 | 83.60±0.49 77.66±0.49 86.69±0.25 | 91.23±0.58 87.04±0.68 94.13±0.33 | 90.44±0.62 84.56±0.53 93.74±0.21 | 95.83±0.27 90.01±0.82 98.12±0.19 | 92.49±0.38 87.71±0.77 94.89±0.31 | 98.12±0.41 88.64±0.73 97.74±0.34 | | LLM Based | Galactica Chem T5 MolCA MolT5 MolTC | 79.28±0.34 85.93±0.31 87.85±0.49 89.59±0.37 95.89∗±0.13 | 86.25±0.34 92.07±0.37 94.01±0.37 93.18±0.37 98.87±0.31 | 67.30±0.56 72.44±0.39 68.51±0.62 76.89±0.41 89.39∗±0.13 | 79.02±0.59 89.51±0.32 88.33±0.52 87.62±0.53 95.55∗±0.17 | 74.51±0.43 80.99±0.52 90.17±0.45 80.97±0.37 95.59∗±0.21 | 84.01±0.67 85.35±0.48 93.04±0.61 90.18±0.38 98.06±0.19 | 71.20±0.42 75.78±0.63 82.97±0.57 89.22±0.37 96.70∗±0.26 | 79.08±0.40 84.52±0.44 89.03±0.74 94.02±0.33 98.85±0.42 | | DO-CoLM (Ours) | DO-CoLM (Ours) | 96.43±0.18 | 99.13±0.31 | 92.14±0.28 | 96.03±0.12 | 96.31±0.32 | 98.52±0.16 | 97.24±0.24 | 99.12±0.31 | Table 2: Comparative performance of various methods in quantitative interactive tasks. The best-performing methods are highlighted in bold, while the second-best methods are marked with * for emphasis. | Baseline Model | Baseline Model | FreeSolv - MAE | FreeSolv - RMSE | Abraham - MAE | Abraham - RMSE | CompSol - MAE | CompSol - RMSE | CombiSolv - MAE | CombiSolv - RMSE | |------------------|-------------------------------------|--------------------------------------------------------------|--------------------------------------------------------------|--------------------------------------------------------------|--------------------------------------------------------------|-------------------------------------------------------------|--------------------------------------------------------------|--------------------------------------------------------------|-------------------------------------------------------------| | GNN Based | MMGNN D-MPNN GEM CGIB | 0.538±0.031 0.712±0.015 0.608±0.019 0.550±0.011 | 0.904±0.028 1.229±0.029 1.198±0.052 0.922±0.057 | 0.197±0.009 0.494±0.013 0.254±0.006 0.261±0.008 | 0.396±0.012 0.715±0.027 0.537±0.008 0.531±0.009 | 0.170∗±0.005 0.208±0.007 0.205±0.007 0.174±0.005 | 0.302±0.005 0.372±0.008 0.343±0.005 0.313±0.004 | 0.186±0.004 0.492±0.014 0.295±0.009 0.231±0.004 | 0.390∗±0.015 0.905±0.055 0.773±0.019 0.395±0.010 | | ML Based | GOVER SolvBert Uni-Mol SMD | 0.646±0.026 0.592±0.031 0.576±0.061 0.609±0.037 | 1.094±0.045 1.054±0.054 1.009±0.072 1.211±0.036 | 0.367±0.009 0.498±0.008 0.365±0.008 0.398±0.022 | 0.635±0.017 0.687±0.015 0.612±0.024 0.645±0.037 | 0.187±0.006 0.191±0.010 0.197±0.002 0.197±0.006 | 0.381±0.015 0.348±0.008 0.345±0.003 0.349±0.007 | 0.413±0.017 0.436±0.018 0.269±0.005 0.656±0.012 | 0.738±0.035 0.709±0.021 0.671±0.017 1.013±0.031 | | LLM Based | Galactica Chem T5 MolCA MolT5 MolTC | 0.892±0.011 0.812±0.036 0.771±0.035 0.715±0.047 0.497∗±0.013 | 1.398±0.067 1.357±0.057 1.292±0.041 1.115±0.075 0.694∗±0.042 | 0.651±0.008 0.639±0.010 0.584±0.007 0.553±0.009 0.194∗±0.011 | 1.058±0.016 0.915±0.017 0.887±0.011 0.842±0.006 0.389∗±0.010 | 0.597±0.008 0.447±0.008 0.477±0.008 0.486±0.003 0.179±0.006 | 0.862±0.008 0.739±0.010 0.726±0.023 0.701±0.008 0.295∗±0.004 | 0.841±0.021 0.892±0.015 0.638±0.043 0.673±0.031 0.183∗±0.004 | 1.456±0.039 1.308±0.024 1.095±0.037 1.114±0.029 0.452±0.008 | | DO-CoLM (Ours) | DO-CoLM (Ours) | 0.469±0.014 | 0.649±0.030 | 0.175±0.008 | 0.372±0.012 | 0.165±0.007 | 0.281±0.003 | 0.172±0.005 | 0.385±0.012 | Qualitative Prediction Performance: Table 1 presents the comparative performance of DO-CoLM and various baseline methods on four widely-used chemical datasets. The results clearly demonstrate the consistent superiority of DOCoLM over existing approaches across all datasets. For instance, on the ZhangDDI dataset, our model achieves a 3% improvement in accuracy compared to MolTC. This can be attributed to the incorporation of our molecular conformational relationship capture mechanism and the heterograph neural network, which effectively integrates conformational information. These enhancements enable the model to capture the 3D structural information and latent conformational interactions that are often overlooked in 1D SMILES and 2D molecular graphs. In addition, the DO-CoLM in our model reorders molecular token nodes from the model's perspective, allowing for a better understanding of molecular modality data and resulting in superior reasoning performance. Quantitative Prediction Performance: Table 2 presents the performance of DO-CoLM on quantitative tasks. The data indicates that LLM-based models generally perform slightly worse than GNN-based and ML-based models. However, DO-CoLM has maintained a leading position in quantitative analysis tasks, which are typically challenging for LLMmodels. DO-CoLM achieved an RMSE of 0.385, representing nearly a 15% improvement over the second-best LLM-based model, MolTC, which had an RMSE of 0.452. Furthermore, DO-CoLM demonstrated significant performance gains over the best GNN-based model, MMGNN. For instance, on the FreeSolv dataset, DO-CoLM reduced the RMSE by nearly 28%. These improvements can be attributed to DO-CoLM's integration of 3D molecular conformation information, which more accurately captures molecular interactions, as well as chain-of-thought reasoning and its unique design considerations for input order from the model's perspective. Table 3: Comparison of results for perturbations in overall external order (EP) and internal multimodal node order (IP). EDRate reflects the performance decrease due to external order perturbations, while IDRate shows the decrease due to internal multimodal node order perturbations. | Datasets | Metrics | DO-CoLM - EP | DO-CoLM - IP | DO-CoLM - EDRate (%) | DO-CoLM - IDRate (%) | MolTC - EP | MolTC - IP | MolTC - RDRate (%) | MolTC - IDRate (%) | |------------|------------------|-----------------------|-----------------------|------------------------|------------------------|-----------------------|-----------------------|----------------------|----------------------| | Drugbank | Accuracy AUC-ROC | 96.16±0.19 98.89±0.30 | 96.33±0.18 99.02±0.31 | 0.27 0.24 | 0.11 0.10 | 84.43±0.17 87.62±0.42 | 94.03±0.13 96.77±0.31 | 12.07 11.37 | 1.93 2.12 | | ZhangDDI | Accuracy AUC-ROC | 92.01±0.27 95.76±0.14 | 92.12±0.28 95.95±0.12 | 0.14 0.15 | 0.02 0.03 | 80.15±0.16 85.64±0.19 | 88.04±0.13 94.37±0.21 | 10.33 10.41 | 1.05 1.19 | | ChChMiner | Accuracy AUC-ROC | 96.06±0.27 98.49±0.16 | 96.14±0.32 98.42±0.15 | 0.25 0.27 | 0.14 0.10 | 82.78±0.17 85.42±0.18 | 93.78±0.21 96.03±0.19 | 13.34 12.89 | 1.89 2.07 | | DeepDDI | Accuracy AUC-ROC | 97.05±0.23 99.01±0.32 | 97.14±0.24 99.03±0.31 | 0.12 0.11 | 0.09 0.10 | 87.62±0.31 89.76±0.37 | 95.46±0.26 96.95±0.42 | 9.45 9.20 | 1.30 1.81 | ## 4.4 Sequence perturbation performance In this section, we test the model's robustness by introducing perturbations to the input data order. Table 3 presents the experimental results of DO-CoLM and MolTC under input sequence perturbations. Since input order inherently exhibits asymmetry in quantitative prediction tasks, this evaluation primarily focuses on qualitative prediction tasks. We introduced two types of input perturbations for the experiments: overall molecular input order perturbation and internal molecular conformation node order perturbation. As shown in Table 3, DO-CoLM demonstrates significant robustness in qualitative prediction tasks, with overall molecular input order perturbation having almost no impact on its performance, while MolTC exhibits significant performance degradation. For example, on the DrugBank dataset, MolTC's accuracy drops from 95.89% to 84.43%, a decrease of 12.07%, and AUC-ROC decreases by 11.37%. Under internal molecular conformation node order perturbations, DO-CoLM remains stable, while MolTC experiences an average decline of 1.7%. This is due to the design of the AOAM, which enables the model to adaptively handle different input sequences, resulting in stable performance. ## 4.5 Ablation Study Table 4 presents the results of the ablation experiments, where w/o DCR indicates the removal of the DCRCM. w/o AO represents the exclusion of AOAM, replaced by three alternative ordering methods: random order, degree-based order, and fixed order. By removing the DCRCM, we analyzed the impact of molecular conformational information interaction on model performance. Additionally, by comparing model performance under random, fixed, and degree-based orders, we validated that AOAM identifies the optimal sequence from the model's perspective. The experimental results demonstrate that each module significantly improves model performance. The removal of DCRCM has the most noticeable impact, causing nearly a 10% decrease in performance on quantitative tasks. Furthermore, replacing AOAMwith other prior knowledge-based ordering methods leads to a certain degree of performance degradation. We also performed an ablation study on the conformations, as detailed in the Appendix E. Table 4: Ablation Study of Removing DCRCM (w/o DCR) and Replacing AOAM (w/o AO) with other ordering methods. | Dataset | Metric | w/o DCR | w/o AO - Random | w/o AO - Deg | w/o AO - Fixed | |-----------|-------------------------------------|-----------------------------------------------------------------------------------------------|---------------------------------------------|-----------------------------------------------------|-----------------------------------------------------| | DDI | Accuracy Rate (↓) AUCROC | 1.22±0.14 1.51 % 1.67±0.32 0.55±0.07 0.62±0.09 | 0.63±0.07 0.77 % 0.78±0.08 | 0.44±0.04 0.54 % | 0.56±0.05 0.69 % | | SSI | Rate (↓) MAE Rate (↑) RMSE Rate (↑) | 1.78 % 0.58 % 0.66 % 7.43 % 0.025±0.0070.016±0.0030.011±0.0040.012±0.005 9.43 % 3.72 % 4.52 % | 0.83 % 5.24 % 6.03 % | 3.06 % | 0.017±0.0040.012±0.0020.007±0.0030.010±0.004 4.37 % | | CSI Abs. | MAE Rate (↑) RMSE | 1.07±0.11 8.04 % 1.58±0.20 0.65±0.04 | 0.61±0.03 4.58 % | 0.48±0.04 3.61 % | 0.51±0.05 3.84 % | | CSI Emis. | Rate (↑) MAE Rate (↑) RMSE | 10.18 % 2.07±0.17 11.12 % 2.89±0.28 1.03±0.11 1.10±0.15 | 0.72±0.06 4.64 % 1.05±0.14 5.64 % 1.47±0.12 | 0.60±0.05 3.87 % 0.74±0.10 3.97 % | 4.18 % 0.81±0.11 4.36 % | | CSI Life. | Rate (↑) MAE Rate (↑) | 12.34 % 4.40 % 4.71 % 10.70 % | 6.27 % 3.76 % | 0.074±0.0030.026±0.0040.016±0.0020.020±0.003 2.31 % | 2.89 % | | | RMSE Rate (↑) | 0.095±0.0100.034±0.0080.023±0.0070.029±0.010 11.26 % | 4.02 % | 2.73 % | 3.43 % | ## 5 Conclusion In this paper, we propose DO-CoLM, which incorporates the Dynamic Conformational Relationship Capture Module and the Adaptive Order Adjustment Module to address the challenges of inadequate molecular 3D information and dynamics, as well as sensitivity to input order in existing LM-based MRL models. Experiments across twelve diverse datasets in various domains demonstrate the superiority of our approach over current GNN and LLM-based baselines. This advancement establishes a new standard for integrating multimodal data in LLM-based MRL. The limitations are further discussed in Appendix F. ## Ethical Statement There are no ethical issues. ## Acknowledgments This work was partially supported by the Project of Stable Support for Youth Team in Basic Research Field, CAS (YSBR-005), Anhui Science Foundation for Distinguished Young Scholars (No.1908085J24), Natural Science Foundation of China (No.62072427), Jiangsu Natural Science Foundation (No. BK20191193). ## Contribution Statement Zhuo Chen is the first author; Wenjie Du and Yang Wang are corresponding authors. ## References 1. [Alonso et al., 2006] Hernan Alonso, Andrey A Bliznyuk, and Jill E Gready. Combining docking and molecular dy- namic simulations in drug design. Medicinal research re- views, 26(5):531-568, 2006. 2. [Andrade et al., 2009] Carolina H Andrade, Kerly FM Pasqualoto, Elizabeth I Ferreira, and Anton J Hopfin- ger. Rational design and 3d-pharmacophore mapping of 5- thiourea-substituted α-thymidine analogues as mycobac- terial tmpk inhibitors. Journal of chemical information and modeling, 49(4):1070-1078, 2009. 3. [Berglund et al., 2023] Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. The reversal curse: Llms trained on" a is b" fail to learn" b is a". arXiv preprint arXiv:2309.12288, 2023. 4. [Christofidellis et al., 2023] Dimitrios Christofidellis, Gior- gio Giannone, Jannis Born, Ole Winther, Teodoro Laino, and Matteo Manica. Unifying molecular and textual rep- resentations via multi-task language modelling. arXiv preprint arXiv:2301.12586, 2023. 5. [Cuturi, 2013] Marco Cuturi. Sinkhorn distances: Light- speed computation of optimal transport. Advances in neu- ral information processing systems, 26, 2013. 6. [Deac et al., 2019] Andreea Deac, Yu-Hsiang Huang, Petar Veličković, Pietro Liò, and Jian Tang. Drug-drug adverse effect prediction with graph co-attention. arXiv preprint arXiv:1905.00534, 2019. 7. Du et al., ] Wenjie Du, Shuai Zhang, Zhaohui Cai, Zhiyuan Liu, Junfeng Fang, Jianmin Wang, and Yang Wang. Molecular merged hypergraph neural network for explain- able solvation free energy prediction. Research, 0(ja). 8. [Du et al., 2024] Wenjie Du, Shuai Zhang, Di Wu, Jun Xia, Ziyuan Zhao, Junfeng Fang, and Yang Wang. MMGNN: A molecular merged graph neural network for explainable solvation free energy prediction. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI 2024, Jeju, South Korea, August 3-9, 2024, pages 5808-5816. ijcai.org, 2024. 9. [Edwards et al., 2021] Carl Edwards, ChengXiang Zhai, and Heng Ji. Text2mol: Cross-modal molecule retrieval with natural language queries. In EMNLP (1), pages 595-607. Association for Computational Linguistics, 2021. 10. [Edwards et al., 2022] Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between molecules and natural language. arXiv preprint arXiv:2204.11817, 2022. 11. [Fang et al., 2022] Xiaomin Fang, Lihang Liu, Jieqiong Lei, Donglong He, Shanzhuo Zhang, Jingbo Zhou, Fan Wang, Hua Wu, and Haifeng Wang. Geometry-enhanced molecu- lar representation learning for property prediction. Nature Machine Intelligence, 4(2):127-134, 2022. 12. [Fang et al., 2024] Junfeng Fang, Shuai Zhang, Chang Wu, Zhengyi Yang, Zhiyuan Liu, Sihang Li, Kun Wang, Wen- jie Du, and Xiang Wang. Moltc: Towards molecular relational modeling in language models. arXiv preprint arXiv:2402.03781, 2024. 13. [Fu et al., 2020] Tianfan Fu, Cao Xiao, and Jimeng Sun. Core: Automatic molecule optimization using copy & re- fine strategy. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 638-645, 2020. 14. [Jablonka et al., 2024] Kevin Maik Jablonka, Philippe Schwaller, Andres Ortega-Guerrero, and Berend Smit. Leveraging large language models for predictive chem- istry. Nature Machine Intelligence, 6(2):161-169, 2024. 15. [Lee et al., 2023] Namkyeong Lee, Dongmin Hyun, Gy- oung S. Na, Sungwon Kim, Junseok Lee, and Chanyoung Park. Conditional graph information bottleneck for molec- ular relational learning. In ICML, volume 202 of Proceed- ings of Machine Learning Research, pages 18852-18871. 16. PMLR, 2023. 17. [Li et al., 2023] Zimeng Li, Shichao Zhu, Bin Shao, Xi- angxiang Zeng, Tong Wang, and Tie-Yan Liu. Dsn-ddi: an accurate and generalized framework for drug-drug in- teraction prediction by dual-view representation learning. Briefings in Bioinformatics, 24(1):bbac597, 2023. 18. [Li et al., 2024] Sihang Li, Zhiyuan Liu, Yanchen Luo, Xi- ang Wang, Xiangnan He, Kenji Kawaguchi, Tat-Seng Chua, and Qi Tian. 3d-molm: Towards 3d molecule-text interpretation in language models. In ICLR, 2024. 19. [Liang et al., 2023] Youwei Liang, Ruiyi Zhang, Li Zhang, and Pengtao Xie. Drugchat: Towards enabling chatgpt- like capabilities on drug molecule graphs. ArXiv, abs/2309.03907, 2023. 20. [Lin et al., 2022] Shenggeng Lin, Yanjing Wang, Lingfeng Zhang, Yanyi Chu, Yatong Liu, Yitian Fang, Mingming Jiang, Qiankun Wang, Bowen Zhao, Yi Xiong, et al. Mdf- sa-ddi: predicting drug-drug interaction events based on multi-source drug fusion, multi-source feature fusion and transformer self-attention mechanism. Briefings in Bioin- formatics, 23(1):bbab421, 2022. 21. [Liu et al., 2023] Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, and Tat- Seng Chua. Molca: Molecular graph-language modeling with cross-modal projector and uni-modal adapter. arXiv preprint arXiv:2310.12798, 2023. 22. [Meng et al., 2023] Fanwang Meng, Hanwen Zhang, Juan Samuel Collins-Ramirez, and Paul W. Ayers. Something for nothing: Improved solvation free energy prediction with learning. 2023. 23. [Morris et al., 2019] Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamilton, Jan Eric Lenssen, Gau- rav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceed- ings of the AAAI conference on artificial intelligence, vol- ume 33, pages 4602-4609, 2019. 24. [Mortier et al., 2015] Jérémie Mortier, Christin Rakers, Mar- cel Bermudez, Manuela S Murgueitio, Sereina Riniker, and Gerhard Wolber. The impact of molecular dynam- ics on drug design: applications for the characterization of ligand-macromolecule complexes. Drug discovery today, 20(6):686-702, 2015. 25. [Nyamabo et al., 2021] Arnold K Nyamabo, Hui Yu, and Jian-Yu Shi. Ssi-ddi: substructure-substructure interac- tions for drug-drug interaction prediction. Briefings in Bioinformatics, 22(6):bbab133, 2021. 26. [Qi et al., 2017] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652-660, 2017. 27. [Rong et al., 2020] Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. Self-supervised graph transformer on large-scale molecular data. In NeurIPS, 2020. 28. [Ryu et al., 2018] Jae Yong Ryu, Hyun Uk Kim, and Sang Yup Lee. Deep learning improves prediction of drug- drug and drug-food interactions. Proceedings of the na- tional academy of sciences, 115(18):E4304-E4311, 2018. 29. [Sagawa and Kojima, 2023] Tatsuya Sagawa and Ryosuke Kojima. Reactiont5: a large-scale pre-trained model to- wards application of limited reaction data. arXiv preprint arXiv:2311.06708, 2023. 30. [Sizhe Liu et al., 2024] Sizhe Sizhe Liu, Jun Xia, Lecheng Zhang, Yuchen Liu, Yue Liu, Wenjie Du, Zhangyang Gao, Bozhen Hu, Cheng Tan, Stan Z Li, et al. Flexmol: A flex- ible toolkit for benchmarking molecular relational learn- ing. Advances in Neural Information Processing Systems, 37:35454-35467, 2024. 31. [Skjærven et al., 2011] Lars Skjærven, Nathalie Reuter, and Aurora Martinez. Dynamics, flexibility and ligand- induced conformational changes in biological macro- molecules: a computational approach. Future Medicinal Chemistry, 3(16):2079-2100, 2011. 32. [Tan et al., 2024] Zhijie Tan, Xu Chu, Weiping Li, and Tong Mo. Order matters: Exploring order sensitivity in multimodal large language models. arXiv preprint arXiv:2410.16983, 2024. 33. [Tang et al., 2024] Xiangru Tang, Andrew Tran, Jeffrey Tan, and Mark B. Gerstein. Mollm: A unified language model for integrating biomedical text with 2d and 3d molecular representations. bioRxiv, 2024. 34. [Taylor et al., 2022a] Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Sar- avia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085, 2022. 35. [Taylor et al., 2022b] Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Sar- avia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085, 2022. 36. [Vermeire and Green, 2021] Florence H Vermeire and William H Green. Transfer learning for solvation free energies: From quantum chemistry to experiments. Chemical Engineering Journal, 418:129307, 2021. 37. [Vinyals et al., 2015] Oriol Vinyals, Samy Bengio, and Man- junath Kudlur. Order matters: Sequence to sequence for sets. arXiv preprint arXiv:1511.06391, 2015. 38. [Wu et al., 2022] Fang Wu, Shuting Jin, Yinghui Jiang, Xu- rui Jin, Bowen Tang, Zhangming Niu, Xiangrong Liu, Qiang Zhang, Xiangxiang Zeng, and Stan Z Li. Pre- training of equivariant graph matching networks with con- formation flexibility for drug binding. Advanced Science, 9(33):2203796, 2022. 39. [Yu et al., 2023] Jiahui Yu, Chengwei Zhang, Yingying Cheng, Yun-Fang Yang, Yuan-Bin She, Fengfan Liu, Weike Su, and An Su. Solvbert for solvation free energy and solubility prediction: a demonstration of an nlp model for predicting the properties of molecular complexes. Dig- ital Discovery, 2(2):409-421, 2023. 40. [Zhang et al., 2025] Shuai Zhang, Junfeng Fang, Xuqiang Li, hongxin xiang, ALAN XIA, Ye Wei, Wenjie Du, and Yang Wang. Iterative substructure extraction for molec- ular relational learning with interactive graph information bottleneck. In The Thirteenth International Conference on Learning Representations, 2025. 41. [Zheng et al., 2025] Yizhen Zheng, Huan Yee Koh, Jiaxin Ju, Anh TN Nguyen, Lauren T May, Geoffrey I Webb, and Shirui Pan. Large language models for scientific dis- covery in molecular property prediction. Nature Machine Intelligence, pages 1-11, 2025. 42. [Zhong et al., 2024] Yujie Zhong, Guangming Li, Jianxin Yang, et al. Learning motif-based graphs for drug-drug interaction prediction via local-global self-attention. Na- ture Machine Intelligence, 6:1094-1105, 2024. 43. [Zhou et al., 2023] Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: a universal 3d molecular representation learning framework. 2023.