---
title: "Dynamic Many-Objective Molecular Optimization: Unfolding Complexity with Objective Decomposition and Progressive Optimization"
authors:
  - "Dong-Hee Shin"
  - "Young-Han Son"
  - "Deok-Joong Lee"
  - "Ji-Wung Han"
  - "Tae-Eui Kam"
doi: "10.24963/ijcai.2024/666"
url: "https://ijcai.org/proceedings/2024/666"
pdf: "https://ijcai.org/proceedings/2024/0666.pdf"
venue: "Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence"
year: 2024
pages: "6026-6034"
track: "Main Track"
keywords:
  - "Multidisciplinary Topics and Applications: MTA: Health and medicine"
  - "Multidisciplinary Topics and Applications: MTA: Bioinformatics"
---

# Dynamic Many-Objective Molecular Optimization: Unfolding Complexity with Objective Decomposition and Progressive Optimization

Dong-Hee Shin, Young-Han Son, Deok-Joong Lee, Ji-Wung Han, Tae-Eui Kam

*Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence*, pages 6026-6034. DOI: [10.24963/ijcai.2024/666](https://doi.org/10.24963/ijcai.2024/666)

## Abstract

Molecular discovery has received significant attention across various scientific fields by enabling the creation of novel chemical compounds. In recent years, the majority of studies have approached this process as a multi-objective optimization problem. Despite notable advancements, most methods optimize only up to four molecular objectives and are mainly designed for scenarios with a predetermined number of objectives. However, in real-world applications, the number of molecular objectives can be more than four (many-objective) and additional objectives may be introduced over time (dynamic-objective). To fill this gap, we propose DyMol, the first method designed to tackle the dynamic many-objective molecular optimization problem by utilizing a novel divide-and-conquer approach combined with a decomposition strategy. Additionally, we comprehensively integrate convergence, Pareto diversity, and structural diversity into the optimization process to provide efficient exploration of the search space. We validate the superior performance of our method using the practical molecular optimization (PMO) benchmark. The source code and supplementary material are available online.

## Full text

## 1 Introduction

Molecular discovery is foundational to progress in a variety of scientific fields, ranging from the development of new pharmaceuticals to the creation of innovative materials [Bilodeau et al., 2022]. Basically, molecular discovery is a complex process that seeks to identify molecules with desirable properties [Son et al., 2024]. In essence, this process is fundamentally a constrained multi-objective optimization problem, where the objectives are to simultaneously maximize or minimize certain attributes of molecules [Fromer and Coley, 2023].

Unlike single-objective optimization, the multi-objective optimization problem introduces distinct challenges that arise from the necessity to balance multiple and often conflicting objectives [Marler and Arora, 2004]. Therefore, it becomes infeasible to identify a single optimal solution that satisfies all

https://github.com/MolecularTeam/DyMol objectives. Instead, the focus shifts to finding Pareto optimal solution sets that represent various trade-offs among these objectives [Gunantara, 2018].

In the context of molecular discovery, the application of multi-objective optimization may exhibit unique characteristics compared to its use in other general domains. First, oracle calls in real-world molecular discovery are expensive, requiring resource-intensive wet-lab experiments or computer simulations to accurately evaluate molecular properties [Huang et al., 2021]. Second, the discrete nature of molecules results in a complex and challenging optimization landscape [Wang et al., 2006]. Lastly, the non-gradual transitions in molecular structures introduce additional complexity by creating an optimization landscape with sharp cliffs [Aldeghi et al., 2022].

To tackle the multi-objective molecular optimization (MOMO) problem, much prior work has employed a range of generative models, including sampling-based methods [Fu et al., 2021; Xie et al., 2021], genetic algorithms [Jensen, 2019; Tripp et al., 2021], probabilistic models [Bengio et al., 2021], and reinforcement learning [Olivecrona et al., 2017; Jin et al., 2020]. However, given the necessity of simultaneously optimizing multiple objectives, they have commonly adopted two multi-objective optimization techniques: scalarization [Eichfelder, 2009] and Bayesian optimization [Laumanns and Ocenasek, 2002]. The scalarization method transforms multiple objectives into a single objective function by aggregating them using weighted sums or other combining strategies [Gunantara, 2018]. On the other hand, the Bayesian optimization method can address multiple objectives concurrently by leveraging acquisition functions to navigate the optimization landscape without needing to quantify the relative weights of each objective [Fromer and Coley, 2023]. While prior frameworks have shown effectiveness in molecular optimization, their applications exhibit distinct constraints. Specifically, they are typically limited to optimizing up to four objectives and they are designed to work with a fixed number of objectives, thereby lacking the capability to adapt to scenarios with varying numbers of objectives.

In real-world applications such as drug discovery, the importance of a dynamic many-objective molecular optimization setting becomes particularly evident [Luukkonen et al., 2023]. From a many-objective perspective, the drug development process is inherently complex and multifaceted, typically requiring optimization of more than four objectives. In particular, a new drug must meet various criteria, including potency, bioavailability, safety, solubility, stability, and synthesizability [Luukkonen et al., 2023]. Concurrently, from a dynamic-objective standpoint, regulatory agencies such as the FDA consistently update their drug approval standards and requirements in response to new scientific insights, public health needs, and safety concerns [Darrow et al., 2020]. These updates can introduce new optimization objectives within the drug development process. However, to our knowledge, no prior studies have tackled these issues.

Figure 1: (a) Pareto front is a set of optimal solutions representing the best possible trade-offs among objectives. Convergence pushes the Pareto front's boundary, while Pareto diversity pulls them apart for a broader spread. (b) Structural diversity, represented by colors, illustrates the variety in molecular scaffolds of chemical compounds.

![Image](https://ijcai.org/proceedings/2024/fulltext/666/image_000000_0c133db9b78cfa95b6de8317080dde6af99608c918c0757c666f303ddca0ea98.png)

The dynamic many-objective molecular optimization problem presents distinct challenges that set it apart from the typical MOMO problem. The many-objective aspect introduces an enormously large search space, which hinders efficient exploration and convergence to optimal Pareto solutions [Yuan et al., 2015]. The dynamic-objective nature of this problem further exacerbates difficulties by continually altering the optimization landscape through the introduction of new objectives. To tackle these challenges, our approach diverges from most previous works, which typically aim to find Pareto optimal solutions for all objectives in a high-dimensional joint search space. Instead of tackling all objectives jointly, we propose a divide-and-conquer approach that decomposes the complex many-objective task into a series of manageable subproblems, allowing us to address each of them sequentially. This progressive optimization scheme systematically unfolds the inherent complexities associated with the many-objective settings. Moreover, we incrementally introduce objectives over time in our method, allowing it to seamlessly adapt to the dynamic-objective settings with the incorporation of new objectives. To enhance our method's adaptability for new objectives, we have also developed an objective adaptation technique that detects changes in the optimization landscape and helps the model to identify effective Pareto solutions.

To further promote the achievement of effective Pareto solutions, we propose the Pareto sampling to strategically emphasize both convergence and Pareto diversity during the optimization process. As shown in Figure 1, convergence refers to the ability of the optimization process to approach the true Pareto optimal front, indicating the proximity of the solutions to the best trade-offs among objectives [Yuan et al., 2015].

On the other hand, Pareto diversity indicates the spread of solutions across the Pareto front, ensuring a wide range of solutions with varying trade-offs among objectives [Yuan et al., 2015]. Pareto diversity is particularly important in molecular optimization, where the optimal trade-off is often unknown in advance [Fromer and Coley, 2023]. In this paper, we comprehensively address both convergence and Pareto diversity through progressive optimization using the divide-andconquer approach alongside convergence and Pareto sampling. In addition to Pareto diversity, we also take into account for molecular structural diversity, which refers to the variety in chemical structures within the generated molecules. Promoting structural diversity is vital in molecular discovery fields such as drug design, as it offers multiple candidate molecules that meet desired criteria while exhibiting distinct chemical structures [Mathur and Hoskins, 2017].

The main contributions of our method can be summarized:

- To the best of our knowledge, DyMol, our proposed method, is the first to tackle the dynamic many-objective molecular optimization problem in molecular discovery.
- We propose a novel divide-and-conquer approach to decompose many-objective problems into sub-problems by starting with a single objective, then systematically adding over time to enable progressive optimization.
- Due to the incremental nature of adding objectives, our approach can handle dynamic-objective settings, where new objectives are introduced throughout the optimization process. In addition, we develop an objective adaptation technique that aids our model in adjusting more easily and effectively to newly introduced objectives.
- As far as we are aware, our method is the first to comprehensively integrate convergence, Pareto diversity, and structural diversity into molecular optimization by utilizing both convergence sampling and Pareto sampling.

## 2 RelatedWork

## 2.1 Generative Models for Molecular Discovery

In recent years, there has been a growing interest in the use of various generative models for molecular discovery. Generative models employed in molecular discovery can be broadly classified into four categories: 1) sampling-based methods, 2) genetic algorithms, 3) reinforcement learning (RL), and 4) probabilistic models. The sampling-based methods [Xie et al., 2021; Fu et al., 2021] focus on sampling from a distribution of possible molecules with desirable properties. The genetic algorithms [Jensen, 2019; Tripp et al., 2021] employ a population-based approach that evolves molecules through iterative selection, crossover, and mutation guided by a fitness function. The RL-based methods [Olivecrona et al., 2017; Jin et al., 2020] involve an agent that interacts with an environment to generate molecular structures. In this paradigm, the agent receives rewards for taking actions that lead to desirable outcomes, thereby gradually refining its strategy through trial and error to learn an optimal policy [Shin et al., 2024]. The probabilistic models, GFlowNets [Bengio et al., 2021], generate molecular structures by identifying high-potential regions using probability distributions learned from data.

## 2.2 Multi-Objective Molecular Optimization

In the context of the MOMO problem, the challenge lies in simultaneously optimizing multiple molecular objectives, which often conflict with each other [Luukkonen et al., 2023]. To address this, two prominent multi-objective optimization techniques have been widely adopted: scalarization and Bayesian optimization (BO). For instance, in the case of scalarization, MIMOSA [Fu et al., 2021] has employed straightforward linear scalarization techniques to handle the MOMO problem. These techniques aim to aggregate multiple objectives into a single objective function by using weighted sums or Tchebycheff methods [Lin et al., 2022]. On the other hand, BO offers a black box optimization approach that has been integrated into various molecular generative models to enhance sample efficiency [Laumanns and Ocenasek, 2002]. In particular, GPBO [Tripp et al., 2021] exemplifies the integration of BO within the framework of GraphGA [Jensen, 2019] as the backbone model. Similarly, LaMBO [Stanton et al., 2022] leverages BO on top of denoising autoencoders to address multi-objective sequence design problems. Recently, HN-GFN [Zhu et al., 2023] proposes a multi-objective BO algorithm that leverages hypernetwork-based GFlowNets.

## 2.3 Dynamic Many-Objective Optimization

Dynamic many-objective optimization, while yet to be widely explored in molecular optimization, has found application in diverse fields such as manufacturing [Quan et al., 2022], environmental management [Liu et al., 2021], and mineral processing [Ding et al., 2018]. Existing approaches in these domains have predominantly employed decomposition-based MOEA/D [Zhang and Li, 2007] and non-dominated sorting NSGA-III [Deb and Jain, 2013] frameworks due to their effectiveness in navigating high-dimensional search space.

## 3 Preliminary

## 3.1 ProblemFormulation

In this work, we tackle the dynamic many-objective molecular optimization problem, characterized by the introduction of new objectives during the optimization process. In this context, 'many-objective' implies optimizing over four or more objectives, presenting a significant increase in complexity and dimensionality compared to a typical multi-objective problem [Hughes, 2005]. The dynamic many-objective molecular optimization problem can be formally defined as:

<!-- formula-not-decoded -->

where x denotes the molecule vector, with X representing the feasible set in n-dimensional solution space. The F(x, t) represents the set of molecular objective functions at time stage t ∈ T. The objective function fi : X × T → R maps the molecular space to the real numbers. Constraints are twofold: inequality constraints gj(x, t) set the boundaries for feasible solutions, while equality constraints hj (x, t) specify exact conditions that feasible solutions must satisfy. The goal of this problem is to identify and track the evolving Pareto front.

## 3.2 Key Pareto Principles in Optimization

As mentioned earlier, the Pareto front is a set of solutions that are each Pareto optimal, which indicates that none of these solutions can be dominated by any other solutions. Therefore, we have the following definitions in our problem formulation:

Definition 1 (Pareto Dominance). Let xa , xb ∈ X , xa is said to Pareto dominate xb (denoted as xa ≻ xb ) if and only if:

<!-- formula-not-decoded -->

Here, fi(xa ) and fi(xb ) represent the values of the ith objective function for solutions xa and xb , respectively.

Definition 2 (Pareto Optimality). A solution x∗ ∈ X is defined as Pareto optimal if no other solution in the feasible set dominates it such as follows:

<!-- formula-not-decoded -->

Definition 3 (Pareto Front). The Pareto front, denoted as PF, indicates the boundary of best possible trade-offs within the objective space. It is formally defined as:

<!-- formula-not-decoded -->

where f(x∗ ) denotes the vector of objective function values corresponding to a Pareto optimal solution.

## 4 Methods

## 4.1 Objective Decomposition

As shown in Figure 2, at time stage t0, our method begins by decomposing complex many-objective sets into more manageable sub-problems. This decomposition process is facilitated by our decomposition module, which analyzes the complexities of each objective and automatically determines the order in which they should be prioritized during optimization. Within this decomposition module, the generative model initially optimizes all objectives jointly for a limited number of iterations. The model uses an initial molecule as a starting point and produces an optimized molecule. Once complete, both the optimized and initial molecules are evaluated by objective functions F(x, t0), also referred to as oracle functions.

From this evaluation, we calculate ordering scores by subtracting the objective scores of the initial molecules from the optimized molecules. These ordering scores offer insights into the extent of improvement in objective scores accomplished by the model. A lower ordering score indicates that optimizing a specific objective is more challenging, implying the need for early prioritization. It is worth noting that if a preferred order for the optimization process is available, this can be employed as the decomposition order as well.

## 4.2 Progressive Optimization

The main idea behind our method is to employ a divide-andconquer approach to provide adaptability and efficiency when dealing with dynamic many-objective optimization. Consider a scenario where we have a total of five molecular objectives: objectives A through E, as shown in Figure 2. Following the decomposition order of A→D→E→B→C, our proposed model begins by solely optimizing objective A as follows:

Figure 2: The overview of our DyMol method for dynamic manyobjective molecular optimization problem, specifically exemplifies the scenario involving five molecular objectives. (a) Our method begins by decomposing complex many-objective sets into more manageable sub-problems facilitated by our decomposition module. (b) The optimization then progresses from a single objective, systematically incorporating additional objectives over time according to the decomposition order, thereby enabling progressive optimization.

![Image](https://ijcai.org/proceedings/2024/fulltext/666/image_000001_c9ddd1fefe2117dac1a8a3935c94d9a95c4599d1079d014e486d571084c272d3.png)

<!-- formula-not-decoded -->

where fA(x) denotes the value of objective A for a molecule x at time stage t1, and X1 represents the feasible set in the decision space specific to t1, potentially different from the general decision space X due to the dynamic nature of the problem. The gj(x, t1) and hj(x, t1) represent inequality and equality constraints, respectively, derived from the physical, biological, or chemical requirements that a molecule must meet to be viable in a real-world environment.

When the model satisfies a certain score threshold related to objective A or reaches a predetermined number of iterations, it progresses to the next time stage t2 and incrementally incorporates additional objective D. Consequently, the optimization problem is expanded to maximize:

<!-- formula-not-decoded -->

However, the introduction of a new objective function fD(·) necessarily alters the optimization landscape by expanding the dimensions of the objective space. In this context, our objective adaptation technique plays a crucial role by enabling the model to adapt to this evolving optimization landscape. Specifically, it detects changes in the composition of objective scores, which provide learning feedback for model training and updates. For instance, at stage t1, the objective scores are solely based on the value of objective A. However, at t2, they evolve to encompass a composite value of both objectives A and D. The major role of the objective adaptation technique is to retrain the model using these updated objective scores, enabling the model to adjust to the evolving Pareto front, which is defined as:

<!-- formula-not-decoded -->

As time stages progress, the model systematically incorporates each new subsequent objective in line with the decomposition order and sequentially adjusts to the evolving Pareto front. Eventually, at the end of the time stage, the model can address the complete set of objectives. Thus, our method can be considered as a divide-and-conquer approach, as it strategically divides the complex optimization task into a series of simpler sub-problems, each focusing on a specific subset of the objectives. However, distinct from conventional divideand-conquer methods that solve sub-problems independently and then combine their solutions, our approach is characterized by its sequential adaptation and refinement of solutions. As new objectives are introduced, the model dynamically adjusts its search process and integrates the incremental sub-problem solutions into a comprehensive solution that addresses all objectives. This adaptive nature of our method can make it effective in the dynamic-objective settings, where the optimization landscape progressively evolves over time.

## Theoretical Analysis on Divide-and-Conquer

Here, we present a theoretical analysis demonstrating that our divide-and-conquer method, despite being incremental in nature, progressively converges toward solutions to those of the original complex problem that addresses all objectives jointly.

Theorem 1 (Convergence of the Divide-and-Conquer Approach to Global Near-Optimal Solutions). Let P be an original problem with objectives {f1, ..., fi, ..., fn}, and Pi be a sub-problem of P focusing on {f1, ..., fi}. The solution {x∗1 , x∗2 , ..., x∗i } obtained at each stage i can be served as effective initial points for the next stage i + 1 to progressively converge towards a globally near-optimal solution for P.

Lemma 1 (Initial Convergence). For the initial base case of the divide-and-conquer approach where i = 1, the solution x∗1 for the sub-problemP1 can converge to a near-optimal solution for the objective f1 even in cases of non-convexity. Let x∗1 = argmaxx∈X1 f1(x). Then, by the Bolzano-Weierstrass theorem [Brattka et al., 2012], there exists a sequence {xk} ⊂ X1 such that xk → x∗1 , and liminfk→∞ f1(x∗1 )-f1(xk) ∥x∗1 -xk∥ ≥ 0.

Figure 3: Reward scores are derived as a weighted sum of objective scores from optimized molecules. Molecules and their rewards are stored in Experience replay, which serves two major roles: ranking molecules by rewards for convergence sampling, and extracting the Pareto front for Pareto sampling. These sampled molecules are then integrated into the loss function for training the generative model.

![Image](https://ijcai.org/proceedings/2024/fulltext/666/image_000002_db5e9df802b4690d1ed8ca79acd89a2661675e80acd49f2ddf27da0e2dd812e8.png)

Lemma 2 (Effective Initial Points). Let ˜x1 be identified as a near-optimal solution for f1 in P1. Extend the optimization problem to P2 by incorporating an additional objective function f2. Utilize ˜x1 as an initial point to define the restricted Pareto set S by:

<!-- formula-not-decoded -->

where f2 is conditioned on ˜x1 and S is initially formed by maximizing f2 while fixing f1 at the near-optimal level achieved by ˜x1. Next, apply the notion of coordinate ascent [Tseng and others, 1988] by re-optimizing f1 under the constraints imposed by the updated f2 values. This process can be iterated until convergence. Let x′ ∈ PF∗2 be a point on the true Pareto front for P2. Assume f1 and f2 satisfy a Lipschitz condition [Hager, 1979] on X2 that there exists at least local Lipschitz constants Ci ≥ 0:

<!-- formula-not-decoded -->

Under the conjecture that if ∥x-x′ ∥ is sufficiently small (implying x and x′ are close to each other), then ∥fi(x)-fi(x′ )∥ is also small. Thus, with the notion of coordinate ascent, S can progressively approach closer to PF∗2 in the input space. Due to page constraints, additional proofs and their proof sketches are provided in the supplementary material 7.1.

## 4.3 Pareto Sampling and Objective Adaptation

Our proposed model is designed to generate molecules that satisfy given objectives, while promoting score-convergence, Pareto diversity, and structural diversity. Our model employs likelihood Pθ to autoregressively generate molecule xg, from initial molecule sequence x0 up to a maximum length L as:

<!-- formula-not-decoded -->

These generated molecules are evaluated by F(x, t) to obtain objective scores. To provide learning feedback, we define the reward scores R(x, t) at each time stage t by computing the weighted sum of objective scores from F(x, t) such as:

<!-- formula-not-decoded -->

where n is the total number of objective functions and wi(t) denotes the relative weight for fi(x) at time stage t.

To further enhance the training efficiency, we utilize experience replay B that stores previously optimized molecules with high reward scores. In contrast to traditional approaches that primarily emphasize score-convergence, we develop the Pareto sampling technique to also consider Pareto diversity.

Specifically, we perform two types of sampling: convergence sampling, where we sample molecules xc with high reward scores from B to promote score-convergence, and Pareto sampling, where we sample molecules xp from the Pareto front to encourage Pareto diversity. Finally, the generative model parameters θ are optimized by the following loss function:

L(θ, t) = [- log Pθ(x) + log Pprior(x) + R(x, t)]2 , (12) where Pprior is the likelihood of a pre-trained model that imposes additional constraints based on the chemical grammar. It should be noted that x encompasses a set of molecules xg, xc, and xp, represented as x = {xg, xc, xp}.

As time stages advance t → t + 1, the introduction of new objective changes the composition of objective scores and the reward scores. Although the generative model is initially unaware of these changes, we introduce the objective adaptation technique to update θ. This involves retraining the model using updated reward scores to account for the impact of new objectives. The objective adaptation loss can be expressed as:

<!-- formula-not-decoded -->

where xb denotes all molecules from B. Note that we employ REINVENT [Olivecrona et al., 2017] as our backbone generative model due to its superior performance. The pseudo-code for the entire process is in the supplementary material 7.2.

## 5 Experiments

## 5.1 Experimental Setup

We evaluated the performance of our proposed method using the practical molecular optimization (PMO) benchmark [Gao et al., 2022]. In this setup, oracle call budgets are strictly limited to 10,000 evaluations to reflect the real-world constraints of molecular discovery. For the oracle functions in our experiments, we adopted the most commonly used molecular objective functions in previous MOMO studies [Jin et al., 2020; Xie et al., 2021]. These include biological objectives GSK3β and JNK3, which represent inhibition scores against two target proteins related to Alzheimer's disease, as well as nonbiological objectives like QED and SA that quantify druglikeness and synthesizability, respectively. To extend our approach to many-objective settings, we included further objectives such as DRD2, associated with dopamine receptor binding affinity, as well as Osimertinib MPO and Fexofenadine MPO objectives for discovering new therapeutics that optimize existing drugs with multiple desirable attributes.

Table 1: Performance comparison in many-objective optimization scenarios with Four objectives (GSK3β+JNK3+QED+SA), Five objectives (GSK3β+JNK3+QED+SA+DRD2), and Six objectives (GSK3β+JNK3+QED+SA+DRD2+Osimertinib MPO) using 10 different seeds.

| Method       | Four objectives - HV(↑)   | Four objectives - R2(↓)   | Five objectives - HV(↑)   | Five objectives - R2(↓)   | Six objectives - HV(↑)   | Six objectives - R2(↓)   |
|--------------|---------------------------|---------------------------|---------------------------|---------------------------|--------------------------|--------------------------|
| Random ZINC  | 0.065±0.013               | 4.809±0.243               | 0.005±0.003               | 8.867±0.202               | 0.001±0.000              | 14.456±0.344             |
| SMILES-VAE   | 0.073±0.023               | 4.839±0.435               | 0.004±0.001               | 9.226±0.512               | 0.001±0.000              | 15.109±0.548             |
| GFlowNet     | 0.063±0.011               | 4.952±0.196               | 0.011±0.004               | 9.011±0.314               | 0.004±0.001              | 14.462±0.422             |
| MIMOSA       | 0.082±0.028               | 4.905±0.410               | 0.016±0.012               | 9.195±0.761               | 0.005±0.005              | 15.454±1.701             |
| LaMBO        | 0.123±0.006               | 4.496±0.148               | 0.009±0.001               | 9.302±0.159               | Out-of-memory            | Out-of-memory            |
| HN-GFN       | 0.120±0.000               | 4.013±0.060               | 0.004±9.861               | 9.861±0.096               | 0.002±0.000              | 15.349±0.102             |
| MOEA/D       | 0.176±0.123               | 4.615±1.193               | 0.094±0.052               | 8.105±1.561               | 0.025±0.018              | 15.054±2.028             |
| NSGA-III     | 0.234±0.107               | 3.477±0.837               | 0.071±0.047               | 7.130±1.349               | 0.016±0.020              | 14.366±2.393             |
| GraphGA      | 0.254±0.069               | 3.379±0.666               | 0.100±0.057               | 7.676±1.312               | 0.051±0.030              | 11.814±1.787             |
| GPBO         | 0.275±0.091               | 3.311±0.757               | 0.091±0.031               | 7.670±0.761               | 0.026±0.024              | 12.840±1.811             |
| REINVENT BO  | 0.309±0.021               | 2.795±0.103               | 0.071±0.014               | 7.537±0.620               | 0.033±0.019              | 10.929±1.019             |
| REINVENT     | 0.338±0.030               | 2.770±0.116               | 0.099±0.054               | 7.578±1.187               | 0.062±0.028              | 10.032±0.922             |
| AugMem       | 0.395±0.038               | 2.496±0.192               | 0.090±0.043               | 8.011±0.955               | 0.071±0.072              | 12.472±3.758             |
| DyMol (Ours) | 0.422±0.023               | 2.297±0.095               | 0.247±0.087               | 4.943±0.990               | 0.143±0.056              | 8.842±1.632              |

Figure 4: Average HV improvement curves for the top 8 methods.

![Image](https://ijcai.org/proceedings/2024/fulltext/666/image_000003_a7deda01ac48f721e4c8656753bd0086c60f8705cfe10020cefc60a2224e40a5.png)

## 5.2 CompetingMethods

We compared the performance of our method against a range of competing methods, including Random ZINC [Sterling and Irwin, 2015], SMILES-VAE [G´omez-Bombarelli et al., 2018], MIMOSA [Fu et al., 2021], GFlowNet [Bengio et al., 2021], and GraphGA [Jensen, 2019]. Additionally, we evaluated against BO methods such as GPBO [Tripp et al., 2021], LaMBO [Stanton et al., 2022], and HN-GFN [Zhu et al., 2023]; well-known many-objective optimization algorithms like MOEA/D [Zhang and Li, 2007] and NSGA-III [Verhellen, 2022]; and RL-based methods, including REINVENT [Olivecrona et al., 2017], REINVENT BO [Tripp et al., 2021], and AugMem [Guo and Schwaller, 2024]. Note that REINVENT was acknowledged as the best-performing algorithm for molecular optimization, as evidenced by the PMO benchmark results [Gao et al., 2022]. Hence, as mentioned earlier, we employed REINVENT as our backbone generative model. However, we distinctively developed it for a dynamic many-objective setting. More information on competing methods, experimental settings, and hyperparameter configurations is available in the supplementary material 7.3.

## 5.3 Experimental Results

The performances of our proposed method and the competing methods were assessed by two evaluation metrics: the hypervolume indicator (HV) [Zitzler et al., 2003] and the R2 indicator (R2) [Brockhoff et al., 2012]. The HVmeasures the volume of the objective space dominated by the Pareto front, while the R2 evaluates the quality of a solution set based on user-defined reference points. A higher HV value indicates a better solution set, while a lower R2 value is more desirable. Refer to the supplementary material 7.4 for detailed explanations of evaluation metrics. Note that each experiment was conducted with 10 different seeds to ensure result reliability.

Table 1 presents the HV and R2 performance with standard deviations for each method across many-objective optimization scenarios with different numbers of objectives as follows:

- Four objectives: GSK3β + JNK3 + QED + SA;
- Five objectives: GSK3β + JNK3 + QED + SA + DRD2;
- Six objectives: GSK3β + JNK3 + QED + SA + DRD2 + Osimertinib MPO.

As shown in Table 1, our method outperforms all competing methods across all scenarios. Notably, in scenarios with Five and Six objectives, our method demonstrates a substantial performance improvement. This highlights the effectiveness of our divide-and-conquer approach, which successfully handles the inherent complexity of many-objective problems by decomposing them into manageable sub-problems. However, other competing methods struggle with exponential increases in complexity. Additional experiments for many-objective scenarios are provided in the supplementary material 7.5.

Figure 4 displays the average HV improvement curves for the top 8 methods in Four objective scenarios. As depicted, our method consistently outperforms others after reaching 2500 oracle calls. Genetic-based algorithms like MOEA/D, NSGA-III, and GPBO demonstrate rapid initial performance improvements but typically reach a plateau beyond 3000 oracle calls. In contrast, RL-based algorithms such as REINVENT and AugMem exhibit more consistent and continual improvement over time. Additional HV improvement curves and analyses can be found in supplementary material 7.6.

Table 2: Ablation study for each technique: Pareto Sampling (PS), Divide-and-Conquer (DC), and Objective Adaptation (OA).

| Ablation - PS   | Ablation - DC   | Ablation - OA   |   Four objectives - HV(↑) |   Four objectives - R2(↓) |   Five objectives - HV(↑) |   Five objectives - R2(↓) |   Six objectives - HV(↑) |   Six objectives - R2(↓) |
|-----------------|-----------------|-----------------|---------------------------|---------------------------|---------------------------|---------------------------|--------------------------|--------------------------|
| -               | -               | -               |                     0.338 |                     2.770 |                     0.099 |                     7.578 |                    0.062 |                   10.032 |
| ✔               | -               | -               |                     0.379 |                     2.501 |                     0.103 |                     7.018 |                    0.073 |                   10.033 |
| -               | ✔               | -               |                     0.363 |                     2.692 |                     0.150 |                     6.276 |                    0.101 |                   10.408 |
| -               | ✔               | ✔               |                     0.373 |                     2.621 |                     0.209 |                     5.480 |                    0.105 |                   10.269 |
| ✔               | ✔               | -               |                     0.412 |                     2.321 |                     0.182 |                     5.488 |                    0.122 |                    9.439 |
| ✔               | ✔               | ✔               |                     0.422 |                     2.297 |                     0.247 |                     4.943 |                    0.143 |                    8.842 |

Figure 5: Finetuning performance of the top 4 methods in dynamicobjective scenarios, which introduce a new molecular objective.

![Image](https://ijcai.org/proceedings/2024/fulltext/666/image_000004_233c8265b8556cd69b08ac72c884011a7d1cc78289a8dd2dab556e5db7375f42.png)

## 5.4 Ablation Study

As shown in Table 2, we have conducted an ablation study to investigate the impact of key techniques on the performance of our method: Pareto Sampling (PS), Divide-and-Conquer (DC), and Objective Adaptation (OA). We observed that each of these techniques significantly contributes to improved performance. DC primarily focuses on improving convergence, bringing solutions closer to the optimal Pareto front values. PS enhances performance through emphasis on Pareto diversity. Remarkably, OA leads to substantial performance gains, especially in Five-objective scenarios, highlighting its capability to adapt effectively to newly introduced objectives.

## 5.5 Dynamic-Objective Scenarios

To assess our method's adaptability in dynamic-objective scenarios, we propose a novel experimental setup where a model has initially been fully optimized for a set of Five objectives. Subsequently, a new, sixth objective (Osimertinib MPO) is introduced, requiring additional optimization. Instead of reoptimizing all objectives from scratch, we implement a finetuning approach that leverages the model already optimized for the initial Five objectives, and further optimizing the new objective. As depicted in Figure 5, our method effectively reaches the baseline performance of the Six objectives within fine-tuning 2000 oracle calls and continues to improve beyond that. This achievement can be attributed to our OA technique and the incremental nature of adding objectives within our method. Details on dynamic-objective scenarios and additional experiments are in the supplementary material 7.7.

Figure 6: Visualization of the Pareto front for biological objectives.

![Image](https://ijcai.org/proceedings/2024/fulltext/666/image_000005_50c8d0e2c8f3fdd0b43c997bb60b018b083f9b9b2ab0380062e048fa964ea412.png)

Table 3: Analysis of structural diversity indicated by the number of unique Bemis-Murcko scaffolds (BM) and carbon skeletons (CS).

| Method       |   Four objectives - BM(↑) |   Four objectives - CS(↑) |   Five objectives - BM(↑) |   Five objectives - CS(↑) |   Six objectives - BM(↑) |   Six objectives - CS(↑) |
|--------------|---------------------------|---------------------------|---------------------------|---------------------------|--------------------------|--------------------------|
| REINVENT     |                      5026 |                      2004 |                      6298 |                      3134 |                     7490 |                     3400 |
| AugMem       |                      6796 |                      2557 |                      7131 |                      3566 |                     6918 |                     4161 |
| DyMol (Ours) |                      7210 |                      3952 |                      7134 |                      3970 |                     7664 |                     4251 |

## 5.6 Visualization of the Pareto Front

To evaluate solution quality in our method, we have attempted to visualize the Pareto front. However, visualizing the Pareto front in settings with more than three objectives is challenging due to the human limitations in interpreting high-dimensional data [Tuˇsar and Filipiˇc, 2014]. Thus, we only focus on visualizing biological objectives in Five objective scenarios, as they are considered to be more important in drug discovery [Sun et al., 2022]. Figure 6 illustrates the 2D and 3D Pareto fronts, comparing our method with baseline REINVENTmethod. In both cases, the solutions of our method dominate the baseline method by approaching the optimal Pareto front more closely. Moreover, our method exhibits a broader distribution of solutions along the Pareto front, suggesting a better exploration.

## 5.7 Structural Diversity Analysis

In the realm of drug discovery, structural diversity plays a pivotal role as it substantially enhances the chances of discovering compounds with distinctive and potent biological activities [Walters and Namchuk, 2003]. To quantitatively assess the structural diversity among the molecules generated by our method, we adopted a diversity metric based on the number of unique Bemis-Murcko scaffolds (BM) and corresponding carbon skeletons (CS) [Bemis and Murcko, 1996]. Refer to the supplementary material 7.8 for further details. As shown in Table 3, our method outperforms other methods in terms of BM and CS, indicating a higher average structural diversity.

## 6 Conclusion

In this work, we propose DyMol as a novel and first method to address the dynamic many-objective molecular optimization problem by leveraging the divide-and-conquer approach. DyMol decomposes complex many-objective sets into manageable sub-problems for progressive optimization. Our results demonstrate that DyMol outperforms competing methods in both many-objective and dynamic-objective scenarios. Future work can include extending research to material science.

## Acknowledgments

This work was supported by grants from the Institute of Information &amp; Communications Technology Planning &amp; Evaluation (IITP)-specifically, the Artificial Intelligence Graduate School Program at Korea University (No. 2019-0-00079) and the Development of AI Autonomy and Knowledge Enhancement for AI Agent Collaboration (No. 2022-0-00871). Additionally, this research received funding from the National Research Foundation of Korea (NRF), supported by the Korea government (MSIT) under grant (No. RS202300212498).

## Contribution Statement

Tae-Eui Kam is the corresponding author. Dong-Hee Shin and Young-Han Son contributed equally to this work as cofirst authors. Deok-Joong Lee assisted with figure preparation and data analysis. Ji-Wung Han proofread the manuscript. All authors have read and approved the final manuscript.

## References

1. [Aldeghi et al., 2022] Matteo Aldeghi, David E Graff, Nathan Frey, Joseph A Morrone, Edward O Pyzer-Knapp, Kirk E Jordan, and Connor W Coley. Roughness of molecular property landscapes and its impact on modella- bility. Journal of Chemical Information and Modeling, 62(19):4660-4671, 2022.
2. [Bemis and Murcko, 1996] Guy W Bemis and Mark A Mur- cko. The properties of known drugs. 1. molecular frame- works. Journal of medicinal chemistry, 39(15):2887- 2893, 1996.
3. [Bengio et al., 2021] Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34:27381-27394, 2021.
4. [Bilodeau et al., 2022] Camille Bilodeau, Wengong Jin, Tommi Jaakkola, Regina Barzilay, and Klavs F Jensen. Generative models for molecular discovery: Recent ad- vances and challenges. Wiley Interdisciplinary Reviews: Computational Molecular Science, 12(5):e1608, 2022.
5. [Brattka et al., 2012] Vasco Brattka, Guido Gherardi, and Alberto Marcone. The Bolzano-Weierstrass theorem is the jump of weak kőnig's lemma. Annals of Pure and Applied Logic, 163(6):623-655, 2012.
6. [Brockhoff et al., 2012] Dimo Brockhoff, Tobias Wagner, and Heike Trautmann. On the properties of the r2 indica- tor. In Proceedings of the 14th annual conference on Ge- netic and evolutionary computation, pages 465-472, 2012.
7. [Darrow et al., 2020] Jonathan J Darrow, Jerry Avorn, and Aaron S Kesselheim. Fda approval and regulation of phar- maceuticals, 1983-2018. Jama, 323(2):164-176, 2020.
8. [Deb and Jain, 2013] Kalyanmoy Deb and Himanshu Jain. An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting ap- proach, part i: solving problems with box con- straints. IEEE transactions on evolutionary computation, 18(4):577-601, 2013.
9. [Ding et al., 2018] Jinliang Ding, Cuie Yang, Qiong Xiao, Tianyou Chai, and Yaochu Jin. Dynamic evolutionary multiobjective optimization for raw ore allocation in min- eral processing. IEEE Transactions on Emerging Topics in Computational Intelligence, 3(1):36-48, 2018.
10. [Eichfelder, 2009] Gabriele Eichfelder. Scalarizations for adaptively solving multi-objective optimization problems. Computational Optimization and Applications, 44:249- 273, 2009.
11. [Fromer and Coley, 2023] Jenna C Fromer and Connor W Coley. Computer-aided multi-objective optimization in small molecule discovery. Patterns, 4(2), 2023.
12. [Fu et al., 2021] Tianfan Fu, Cao Xiao, Xinhao Li, Lu- cas M Glass, and Jimeng Sun. Mimosa: Multi-constraint molecule sampling for molecule optimization. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 35, pages 125-133, 2021.
13. [Gao et al., 2022] Wenhao Gao, Tianfan Fu, Jimeng Sun, and Connor Coley. Sample efficiency matters: a bench- mark for practical molecular optimization. Advances in Neural Information Processing Systems, 35:21342-21357, 2022.
14. [Gómez-Bombarelli et al., 2018] Rafael Gómez- Bombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez- Lengeling, Dennis Sheberla, Jorge Aguilera-Iparraguirre, Timothy D Hirzel, Ryan P Adams, and Alán Aspuru- Guzik. Automatic chemical design using a data-driven continuous representation of molecules. ACS central science, 4(2):268-276, 2018.
15. [Gunantara, 2018] Nyoman Gunantara. A review of multi- objective optimization: Methods and its applications. Co- gent Engineering, 5(1):1502242, 2018.
16. [Guo and Schwaller, 2024] Jeff Guo and Philippe Schwaller. Augmented memory: Sample-efficient generative molecu- lar design with reinforcement learning. JACS Au, 2024.
17. [Hager, 1979] William W Hager. Lipschitz continuity for constrained processes. SIAM Journal on Control and Op- timization, 17(3):321-338, 1979.
18. [Huang et al., 2021] Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. Ther- apeutics data commons: Machine learning datasets and tasks for drug discovery and development. arXiv preprint arXiv:2102.09548, 2021.
19. [Hughes, 2005] Evan J Hughes. Evolutionary many- objective optimisation: many once or one many? In 2005 IEEE congress on evolutionary computation, volume 1, pages 222-227. IEEE, 2005.
20. [Jensen, 2019] Jan H Jensen. A graph-based genetic algo- rithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):3567-3572, 2019.
21. [Jin et al., 2020] Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objective molecule generation using in- terpretable substructures. In International conference on machine learning, pages 4849-4859. PMLR, 2020.
22. [Laumanns and Ocenasek, 2002] Marco Laumanns and Jiri Ocenasek. Bayesian optimization algorithms for multi- objective optimization. In International Conference on Parallel Problem Solving from Nature, pages 298-307. Springer, 2002.
23. [Lin et al., 2022] Xi Lin, Zhiyuan Yang, Xiaoyuan Zhang, and Qingfu Zhang. Pareto set learning for expensive multi- objective optimization. Advances in Neural Information Processing Systems, 35:19231-19247, 2022.
24. [Liu et al., 2021] Ruochen Liu, Ping Yang, and Jiangdi Liu. A dynamic multi-objective optimization evolutionary al- gorithm for complex environmental changes. Knowledge- Based Systems, 216:106612, 2021.
25. [Luukkonen et al., 2023] Sohvi Luukkonen, Helle W van den Maagdenberg, Michael TM Emmerich, and Gerard JP van Westen. Artificial intelligence in multi- objective drug design. Current Opinion in Structural Biology, 79:102537, 2023.
26. [Marler and Arora, 2004] R Timothy Marler and Jasbir S Arora. Survey of multi-objective optimization methods for engineering. Structural and multidisciplinary optimiza- tion, 26:369-395, 2004.
27. [Mathur and Hoskins, 2017] Sunil Mathur and Clare Hoskins. Drug development: Lessons from nature. Biomedical reports, 6(6):612-614, 2017.
28. [Olivecrona et al., 2017] Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics, 9(1):1-14, 2017.
29. [Quan et al., 2022] Zhen Quan, Yan Wang, and Zhicheng Ji. Multi-objective optimization scheduling for manufactur- ing process based on virtual workflow models. Applied Soft Computing, 122:108786, 2022.
30. [Shin et al., 2024] Dong-Hee Shin, Young-Han Son, Jun-Mo Kim, Hee-Jun Ahn, Jun-Ho Seo, Chang-Hoon Ji, Ji-Wung Han, Byung-Jun Lee, Dong-Ok Won, and Tae-Eui Kam. MARS: Multiagent reinforcement learning for spatial- spectral and temporal feature selection in EEG-based BCI. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2024.
31. [Son et al., 2024] Young-Han Son, Dong-Hee Shin, and Tae- Eui Kam. FTMMR: Fusion transformer for integrat- ing multiple molecular representations. IEEE Journal of Biomedical and Health Informatics, 2024.
32. [Stanton et al., 2022] Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Greenside, and Andrew Gordon Wilson. Accelerating bayesian optimization for biological sequence design with denoising autoencoders. In International Conference on Machine Learning, pages 20459-20478. PMLR, 2022.
33. [Sterling and Irwin, 2015] Teague Sterling and John J Irwin. Zinc 15-ligand discovery for everyone. Journal of chemi- cal information and modeling, 55(11):2324-2337, 2015.
34. [Sun et al., 2022] Mengying Sun, Jing Xing, Han Meng, Huijun Wang, Bin Chen, and Jiayu Zhou. Molsearch: search-based multi-objective molecular generation and property optimization. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 4724-4732, 2022.
35. [Tripp et al., 2021] Austin Tripp, Gregor NC Simm, and José Miguel Hernández-Lobato. A fresh look at de novo molecular design benchmarks. In NeurIPS 2021 AI for Science Workshop, 2021.
36. [Tseng and others, 1988] Paul Tseng et al. Coordinate as- cent for maximizing nondifferentiable concave functions. 1988. [Tušar and Filipič, 2014] Tea Tušar and Bogdan Filipič. Vi- sualization of pareto front approximations in evolutionary multiobjective optimization: A critical review and the pro- section method. IEEE Transactions on Evolutionary Com- putation, 19(2):225-245, 2014.
37. [Verhellen, 2022] Jonas Verhellen. Graph-based molecu- lar pareto optimisation. Chemical Science, 13(25):7526- 7535, 2022.
38. [Walters and Namchuk, 2003] W Patrick Walters and Mark Namchuk. Designing screens: how to make your hits a hit. Nature reviews Drug discovery, 2(4):259-266, 2003.
39. [Wang et al., 2006] Mingliang Wang, Xiangqian Hu, David N Beratan, and Weitao Yang. Designing molecules by optimizing potentials. Journal of the American Chemical Society, 128(10):3228-3232, 2006.
40. [Xie et al., 2021] Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. Mars: Markov molecular sampling for multi-objective drug discovery. arXiv preprint arXiv:2103.10432, 2021.
41. [Yuan et al., 2015] Yuan Yuan, Hua Xu, Bo Wang, Bo Zhang, and Xin Yao. Balancing convergence and diversity in decomposition-based many-objective optimiz- ers. IEEE Transactions on Evolutionary Computation, 20(2):180-198, 2015.
42. [Zhang and Li, 2007] Qingfu Zhang and Hui Li. MOEA/D: A multiobjective evolutionary algorithm based on decom- position. IEEE Transactions on evolutionary computation, 11(6):712-731, 2007.
43. [Zhu et al., 2023] Yiheng Zhu, Jialu Wu, Chaowen Hu, Ji- ahuan Yan, Chang-Yu Hsieh, Tingjun Hou, and Jian Wu. Sample-efficient multi-objective molecular optimization with GFlownets. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
44. [Zitzler et al., 2003] Eckart Zitzler, Lothar Thiele, Marco Laumanns, Carlos M Fonseca, and Viviane Grunert Da Fonseca. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Transactions on evolutionary computation, 7(2):117-132, 2003.