DO-CoLM: Dynamic 3D Conformation Relationships Capture with Self-Adaptive Ordering Molecular Relational Modeling in Language Models

1 Introduction

Molecular Relationship Learning (MRL) aims to understand interactions between molecular pairs, a critical topic with applications in drug discovery and materials science. For example, drug-drug interactions (DDIs) are vital in pharmacology and drug development, while solute-solvent interactions (SSIs) play a key role in solution chemistry and chemical process design. However, validating these interactions experimentally is time-intensive and costly.

Large Language Models (LLMs), with their vast knowledge base and advanced reasoning capabilities, offer a promising solution [Taylor et al., 2022a]. Models like ReactionT5 [Sagawa and Kojima, 2023] and MolTC [Fang et al., 2024] extend LLMs to MRL by leveraging multimodal data, including molecular graphs (2D), chemical properties, and SMILES (1D), for accurate interaction predictions. Despite their potential, these models face unexplored challenges:

Lack of the molecular stereochemistry and dynamics. Existing LLMs primarily rely on SMILES strings and molecular graphs for predicting molecular interactions [Fang et al., 2024; Sagawa and Kojima, 2023], but these 1D and 2D representations fail to capture the 3D information of molecules [Liu et al., 2023] and the dynamic nature of molecular conformations during chemical reactions [Morris et al., 2019]. Essential properties such as stereochemistry (including chirality and stereoisomerism) and molecular dynamics [Andrade et al., 2009] are crucial for understanding molecular interactions and predicting reaction mechanisms [Alonso et al., 2006]. On the one hand, as shown in Figure 1, a molecule has multiple conformations that continuously interconvert [Skjærven et al., 2011]; on the other hand, the interaction between two molecules occurs through the interaction of their specific conformations, determined by their 3D structures [Mortier et al., 2015], such as the binding of small-molecule and target proteins [Wu et al., 2022].

Ignorance of the influence of input order. The performance of LLM is naturally influenced by the input sequence [Berglund et al., 2023], which can have catastrophic consequences in certain MRL tasks. For instance, in DDI tasks, the SMILES order of input molecules should exhibit symmetry, meaning that swapping the SMILES input order should not affect the model's prediction results. However, as shown in Figure 2, swapping the input SMILES order causes the model to produce different predictions, leading to a decrease in MolTC's performance, with accuracy dropping by 11.58% on the Drugbank dataset, 9.24% on the ZhangDDI dataset, 12.81% on the ChChMiner dataset, and 9.08% on the DeepDDI dataset. Recent studies have highlighted this issue [Berglund et al., 2023], which is attributed to the autoregressive training paradigm typically employed by LLMs, leading the model to treat input order as a feature. This can be helpful in some tasks, such as SSI tasks, but it should not affect model performance in symmetric tasks. Notably, in addition to the overall SMILES order, the internal order of molecular tokens also could affect the model's performance.

Considering the above issues, we propose DO-CoLM: a Dynamic relationship capture and self-adaptive Ordering 3D molecular Conformation LM for MRL. Specifically, we design the Dynamic Conformational Relationship Capture Module (DCRCM), to address the lack of molecular stereochemistry and dynamics. This module models both intramolecular conformation transformation and intermolecular conformation interaction relationships using heterogeneous graphs based on conformational information. Additionally, we introduce the Adaptive Order Adjustment Module (AOAM), which ensures the model's robustness to perturbations in symmetric input orders and allows it to adaptively find the optimal sequence for asymmetric input orders from the model's perspective.

The main contributions of this paper could be summarized as follows:

• Given the current limitations of LLM in adequately considering molecular 3D information and dynamics, and the ne-
• glect of input order in MRL tasks, we propose DO-CoLM, a multimodal framework that adaptively integrates conformational information and accommodates molecular order.
• DO-CoLM enables dynamic learning of relationships between molecular conformations by constructing a heterogeneous graph and adaptively selecting the optimal conformation order, thereby achieving comprehensive molecular information capture and tolerance to molecular encoder tokens variations.
• The superiority of DO-CoLM is empirically validated through extensive experiments across 12 datasets from various domains, including DDI, CSI, and SSI tasks. These results demonstrate that our approach significantly outperforms existing methods.

2 RelatedWork

2.1 Molecular Relational Learning

Early research primarily relied on graph neural networks (GNNs) to construct predictive frameworks for MRL [Du et al., ; Fu et al., 2020; Sizhe Liu et al., 2024]. For instance, Nyamabo et al. proposed a substructure-substructure interaction framework [Zhong et al., 2024], which utilizes graph attention network (GAT) layers for substructure extraction and a co-attention layer to model interactions between different substructures. To further capture molecular interactions, Lee et al. [Lee et al., 2023] introduced the Conditional Graph Information Bottleneck (CGIB) model, inspired by the information bottleneck theory. However, their models lack prior knowledge and fail to leverage the advantages of LLMs.

2.2 LLMs in the Molecular Domain

LLMs have been widely applied in the molecular domain [Zheng et al., 2025; Jablonka et al., 2024]. In 1D, model like MolT5 [Edwards et al., 2022] tokenizes SMILES strings; in 2D, methods such as Text2Mol [Edwards et al., 2021], MolCA [Liu et al., 2023], and DrugChat [Liang et al., 2023] combine molecular graphs with LLMs; and in 3D, MolLM [Tang et al., 2024] and 3D-MoLM [Li et al., 2024] capture spatial features using attention and 3D encoders. Multimodal models like ReactionT5 [Sagawa and Kojima, 2023] and MolTC [Fang et al., 2024] integrate 1D, 2D, and chemical property data for prediction. However, most existing work focuses on single molecules or 1D and 2D data, overlooking the role of 3D conformations in MRL.

2.3 Input Order

Previous studies show that input sequence order significantly affects model performance. For instance, the improved Seq2Seq model [Vinyals et al., 2015] reveals that even unordered data can benefit from an optimal input order. PointNet [Qi et al., 2017] mitigates order sensitivity using symmetry functions. Work on the reversal curse [Berglund et al., 2023] and inter-modal order [Tan et al., 2024] further highlights order's impact in LMs. However, most focus on single-modal inputs or coarse inter-modal order, neglecting fine-grained token-level order across modalities.

3 Our ProposedModel: DO-CoLM

In this section, we present the architecture of DO-CoLM. As shown in Figure 3, DO-CoLM consists of five key steps. First, the 3D Encoder is used to obtain molecular conformer feature embeddings. Next, the DCRCM dynamically models the intramolecular and intermolecular conformational relationships. Then, the AOAM adaptively adjusts the input order based on the asymmetry of the input data to optimize the model's performance. Afterward, the Alignment module ensures seamless integration by aligning molecular modality data with textual data. Finally, the LLM generates the model's output based on the processed information.

3.1 Conformer Encoder

Let C a = [C a 1 , . . . , C ana ] and C b = [C b 1 , . . . , C bnb ] represent the conformer pairs for molecule a and molecule b, where C a i and C b i are the i-th conformations, na and nb are the number of conformations for each molecule. We leverage UniMol [Zhou et al., 2023], a powerful 3D molecular feature encoder f, along with attention pooling p, to capture the embeddings of conformer pairs.

where Va and Vb represent the sets of conformation feature embeddings for the two molecules.

3.2 Dynamic Conformational Relationship Capture Module

Due to the dynamic nature of molecular conformations, accurately modeling the transitions and interactions of molecular conformations requires complex simulations and experiments. Therefore, a conformation transition relationship graph based on predefined structures is not feasible. To address this, we propose the DCRCM, which allows the model to dynamically model intramolecular transitions and intermolecular interactions.

We define this relationship as the Heterogeneous Molecular Conformation Interaction Graph (HMCG). The HMCG is represented by GHMCG = (V, E,A, T ,R), where V = Va ∪Vb is the set of conformer nodes, E is the edge set with three types: intra-molecular edges within a, intra-molecular edges within b, and inter-molecular edges between a and b. A is the adjacency matrix representing node connectivity, T denotes node types, and R denotes edge types.

To model the intra-molecular conformation transformation edges Eintra, we first obtain Aintra through the following algorithm, taking molecule a as an example.

where Ua1 represents the representation learning for the intramolecular conformation as the source node, and Ua2 as the target node, θ1 and θ2 represent the learnable vectors for learning the source and target nodes, respectively, and σ1 and σ2 are the activation functions. Such an operation ensures that the generated graph is a directed and asymmetric graph.

To reduce the complexity of the graph, we perform a sparsification operation on it.

where argtopk(·) returns the indices of the most likely conformation nodes for transformation. The weights of other edges are set to 0. For the conformation transformation edges in molecule b, we adopt the same approach to obtain Ab intra .

Since the inter-molecular conformation interaction is symmetric, we use a bidirectional graph to construct the intermolecular conformation interaction edges Einter. The algorithm to obtain the adjacency matrix Ainter representing the edges is as follows:

where Ua and Ub represent the relationship representations of the source node and target node for the intermolecular conformation interaction, respectively. Here, we use molecule a as the source node and molecule b as the target node, and Θa and Θb are learnable vectors for learning the relationship between the source node and the target node. After applying the same sparsification operation as in Equation (3), we obtain Ainter.

After obtaining GHMCG through the above operation, we aggregate the node information on the graph using Attention(·) and Message(·). The specific computation formula is as follows:

Attention(u, e, v)

where N(v) represents the set of nodes connected to node v, F τ(u) denotes a fully connected layer with parameters {Wτ(v) , bτ(v) }, ρ(e) represents the type of edge, ∥ represents the concatenation operation, and nh denotes the number of attention heads. Then update the node information.

where ⊕ represents the aggregation operation, h(l) v is an aggregated feature representation of the conformation.

3.3 Adaptive Order Adjustment Module

Due to the sensitivity of large models to input order, different input orders can significantly impact model performance. In our model, in addition to the overall SMILES order, there is also the internal order of nodes which also impacts the performance of the model. For these two types of order, we have designed different strategies to enable the model to achieve optimal performance.

The overall order of SMILES. In the DDI task, there are only two possible input molecular sequences: molecular a followed by molecular b, and molecular b followed by molecular a. These two sequences are symmetric and should not affect the model's experimental results. Therefore, we can use a simple operation to enable the model to understand these two different sequences, ensuring that the overall order of the input molecules does not influence the model's performance. This can be achieved by randomly shuffling the input sequence, allowing the model to focus solely on the molecular information itself without considering the positional information it carries.

The internal order of nodes. GHMCG comprises n conformer nodes in total, corresponding to n! possible sequences. Achieving order invariance by enabling the model to simultaneously process all these sequences is nearly impossible. Given that different sequences impact model performance to varying degrees, we propose that an optimal sequence exists from the model's perspective, one that cannot be captured through prior knowledge-based ordering. To address this, we hypothesize that each conformer has an inherent priority and allow the model to autonomously determine these priorities and dynamically adjust them based on different situations, thereby uncovering the optimal sequence in the model's view.

We use a Learnable Sorting Network K(·) with a learnable parameter for each conformation to allow the model to prioritize the node hi. Then we obtain the sorted order Sb while the initial order is Sa.

where n is the number of the conformers. Due to the nondifferentiability of argsort, we employed the Sinkhorn Operator [Cuturi, 2013] to dynamically update the sorting process. Initially, we obtain the transport matrix Q.

where Sia and Sj b represent the position values of conformers i and j, ϵ is a control factor. In the iterations, Q is updated as:

where K is the size of Q, which means the number of rows and columns, u = 1K. The iterations stop when ∆(vQ⊤u, 1K/K) < η. The max iteration number is depend on ϵ: typically, the smaller ϵ, the larger the number is needed to ensure that vQ⊤u is close to 1K/K. Then we obtain the conformer representation Z by normalizing Q and multiplying with the conformation H.

where diag(·) means diagonal operation.

3.4 Alignment

After acquiring the multi-conformer representations Za and Zb of molecule a and molecule b, the next step is to map them into the backbone LLM's hidden space using projectors fproj1, fproj2, where fproj1 and fproj2 share the same parameters. These projectors act as critical connectors, translating Za and Zb into LLM-comprehensible encodings Ma and Mb.

We instantiate fproj1 and fproj2 using trainable projection matrices, which share the same dimensionality as the word embedding space in the language model. More formally, the encodings can be expressed as:

where d denotes the feature dimensionality of the LLM and mi represents the embedded representation of a single aligned conformer.

3.5 Backbone LLM

DO-CoLM uses Galactica1.3B [Taylor et al., 2022b], a decoder-only transformer based on the OPT framework, trained on scientific literature, as its backbone. It excels in interpreting molecular sequences. DO-CoLM leverages Galactica's reasoning to analyze interactions between two molecular sets and their information tokens. The prompt sequence X is as follows:

where n denotes the total integrated input length, P represents the task-specific prompt, and P refers to a collection of manually designed prompts, each specifically crafted for the molecular interaction task r. The details of the prompt will be discussed in Section 3.6. The generation process employs a causal mask to produce a response that encapsulates the key interactive properties, with a length of L:

The training objective is to predict the target response from the input prompt X. Specifically, the output for the i-th token, represented as ˆxi, is determined based on its preceding tokens as follows for t ∈ (1, L):

3.6 Prompt

For different tasks related to molecular interactions, we have designed distinct prompts. Here, SMILES1 and SMILES2 represents the SMILES notation, ConEmb1 and ConEmb2 represents multiple conformer tokens Ma and Mb. For DDI tasks, the goal is to enable the LLM to leverage the conformational information of the two input molecules. By doing so, the LLM actively determines and provides an answer as to whether there is an interaction between the two molecules, drawing on their respective conformations to make judgment.

Prompt for DDI Tasks

For SSI tasks, where LLMs are less adept, we design a prompt that encourages the model to consider molecular conformations during inference. Instead of a direct answer, the model predicts a range of possible interaction outcomes based on the conformations. It then synthesizes the data to estimate a specific value within that range.

Prompt for SSI Tasks

4 Eeperimental Results

4.1 Experimental Setting

We evaluate DO-CoLM on well-established downstream molecule interaction tasks involving qualitative and quantitative analysis. Here we provide a overview of our experimental setup. Detailed descriptions are presented in the Appendix C. Datasets. We employ 12 datasets across various domains such as DDI, SSI, and CSI. Specifically, we collect Drugbank (Version 5.0.3), ZhangDDI, ChChMiner, DeepDDI, TWOSIDES, Chromophore, MNSol, CompSol, Abraham, CombiSolv, FreeSolv, and CombiSolv-QM. Here, we process each molecule using RDKit to obtaining several conformations for each molecule. The hyperparameter experiment on the number of different conformations is shown in the Appendix E. Here, 10 conformations are used in subsequent experiments. Baselines. For a comprehensive evaluation, we conduct various baseline methods encompassing distinct categories such as methods based on: GNNs, DL models other than GNN, and LLMs. Specifically, for DDI task, we employ IGIB-ISE [Zhang et al., 2025], MHCADDI [Deac et al., 2019], DeepDDI [Ryu et al., 2018], SSI-DDI, CGIB, CMRL, MDF-SADDI [Lin et al., 2022], DSN-DDI [Li et al., 2023], MolTC [Fang et al., 2024] as the baseline. For SSI and CSI tasks, we utilize D-MPNN [Vermeire and Green, 2021], SolvBert [Yu et al., 2023], SMD [Meng et al., 2023], CGIB, MMGNN [Du et al., 2024], GEM [Fang et al., 2022], GOVER [Rong et al., 2020], Uni-Mol [Zhou et al., 2023] as the baseline. Furthermore, all downstream tasks adopt LLM-based methods, such as Galactica, Chem T5 [Christofidellis et al., 2023], MolT5, MolCA [Liu et al., 2023] and MolTC as the baseline.

4.2 EvaluationMetrics

We employ prediction Accuracy (%) and AUC-ROC (Area Under the Receiver Operating Characteristic curve) as comparative metrics, while for quantitative tasks, MAE (Mean Absolute Error) and RMSE (Root Mean Square Error) are utilized as the metrics.

4.3 Experimental Results and Analysis

Due to the limitations of certain models on some datasets, we only present a subset of the results here. For detailed results on other datasets, please refer to Appendix D.

Qualitative Prediction Performance: Table 1 presents the comparative performance of DO-CoLM and various baseline methods on four widely-used chemical datasets. The results clearly demonstrate the consistent superiority of DOCoLM over existing approaches across all datasets. For instance, on the ZhangDDI dataset, our model achieves a 3% improvement in accuracy compared to MolTC. This can be attributed to the incorporation of our molecular conformational relationship capture mechanism and the heterograph neural network, which effectively integrates conformational information. These enhancements enable the model to capture the 3D structural information and latent conformational interactions that are often overlooked in 1D SMILES and 2D molecular graphs. In addition, the DO-CoLM in our model reorders molecular token nodes from the model's perspective, allowing for a better understanding of molecular modality data and resulting in superior reasoning performance.

Quantitative Prediction Performance: Table 2 presents the performance of DO-CoLM on quantitative tasks. The data indicates that LLM-based models generally perform slightly worse than GNN-based and ML-based models. However, DO-CoLM has maintained a leading position in quantitative analysis tasks, which are typically challenging for LLMmodels. DO-CoLM achieved an RMSE of 0.385, representing nearly a 15% improvement over the second-best LLM-based model, MolTC, which had an RMSE of 0.452. Furthermore, DO-CoLM demonstrated significant performance gains over the best GNN-based model, MMGNN. For instance, on the FreeSolv dataset, DO-CoLM reduced the RMSE by nearly 28%. These improvements can be attributed to DO-CoLM's integration of 3D molecular conformation information, which more accurately captures molecular interactions, as well as chain-of-thought reasoning and its unique design considerations for input order from the model's perspective.

4.4 Sequence perturbation performance

In this section, we test the model's robustness by introducing perturbations to the input data order. Table 3 presents the experimental results of DO-CoLM and MolTC under input sequence perturbations. Since input order inherently exhibits asymmetry in quantitative prediction tasks, this evaluation primarily focuses on qualitative prediction tasks. We introduced two types of input perturbations for the experiments: overall molecular input order perturbation and internal molecular conformation node order perturbation. As shown in Table 3, DO-CoLM demonstrates significant robustness in qualitative prediction tasks, with overall molecular input order perturbation having almost no impact on its performance, while MolTC exhibits significant performance degradation. For example, on the DrugBank dataset, MolTC's accuracy drops from 95.89% to 84.43%, a decrease of 12.07%, and AUC-ROC decreases by 11.37%. Under internal molecular conformation node order perturbations, DO-CoLM remains stable, while MolTC experiences an average decline of 1.7%. This is due to the design of the AOAM, which enables the model to adaptively handle different input sequences, resulting in stable performance.

4.5 Ablation Study

Table 4 presents the results of the ablation experiments, where w/o DCR indicates the removal of the DCRCM. w/o AO represents the exclusion of AOAM, replaced by three alternative ordering methods: random order, degree-based order, and fixed order. By removing the DCRCM, we analyzed the impact of molecular conformational information interaction on model performance. Additionally, by comparing model performance under random, fixed, and degree-based orders, we validated that AOAM identifies the optimal sequence from the model's perspective. The experimental results demonstrate that each module significantly improves model performance. The removal of DCRCM has the most noticeable impact, causing nearly a 10% decrease in performance on quantitative tasks. Furthermore, replacing AOAMwith other prior knowledge-based ordering methods leads to a certain degree of performance degradation. We also performed an ablation study on the conformations, as detailed in the Appendix E.

5 Conclusion

In this paper, we propose DO-CoLM, which incorporates the Dynamic Conformational Relationship Capture Module and the Adaptive Order Adjustment Module to address the challenges of inadequate molecular 3D information and dynamics, as well as sensitivity to input order in existing LM-based MRL models. Experiments across twelve diverse datasets in various domains demonstrate the superiority of our approach over current GNN and LLM-based baselines. This advancement establishes a new standard for integrating multimodal data in LLM-based MRL. The limitations are further discussed in Appendix F.

Ethical Statement

There are no ethical issues.

Acknowledgments

This work was partially supported by the Project of Stable Support for Youth Team in Basic Research Field, CAS (YSBR-005), Anhui Science Foundation for Distinguished Young Scholars (No.1908085J24), Natural Science Foundation of China (No.62072427), Jiangsu Natural Science Foundation (No. BK20191193).

Contribution Statement

Zhuo Chen is the first author; Wenjie Du and Yang Wang are corresponding authors.