On the Approximation Ability of Evolutionary Optimization with Application to Minimum Set Cover: Extended Abstract

1 Introduction

Evolutionary algorithms (EAs) [B¨ack, 1996] have been successfully applied to many fields and can achieve extraordinary performance in addressing some real-world hard problems, particularly NP-hard problems [Koza et al., 2003; Hornby et al., 2006; Benkhelifa et al., 2009]. To gain an understanding of the behavior of EAs, many theoretical studies have focused on the running time required to achieve exact optimal solutions [He and Yao, 2001; Yu and Zhou, 2008; Neumann and Witt, 2010; Auger and Doerr, 2011]. In practice, EAs are most commonly used to obtain satisficing solutions, yet theoretical studies of their approximation ability have only emerged in the recent decade.

He and Yao (2003) first studied conditions under which the wide-gap far distance and the narrow-gap long distance problems are hard to approximate using EAs. Giel and Wegener (2003) investigated a (1+1)-EA for the maximum matching problem and found that the time taken to find exact optimal solutions is exponential but is only O(n2⌈1/ϵ⌉ ) for (1 + ϵ)-approximate solutions, which demonstrates the value of EAs as approximation algorithms. Subsequently, further results on the approximation ability of EAs were reported. For the (1+1)-EA, the simplest type of EA, two directions of results have been obtained. On one hand, it was found that the (1+1)-EA has an arbitrarily poor approximation ratio for the minimum vertex cover as well as the minimum set cover problem [Friedrich et al., 2010; Oliveto et al., 2009]. Meanwhile, it was also found that (1+1)-EA can be a good approximation algorithm for a subclass of the partition problem [Witt, 2005] and some subclasses of the minimum vertex cover problem [Oliveto et al., 2009], and can be a useful postoptimizer [Friedrich et al., 2009].

∗This paper is an extended abstract of the AI Journal publication [Yu et al., 2012]. Full proofs can be found in the journal paper.

Recent advances in multi-objective EAs have shed light on the power of EAs as approximation optimizers. For a single-objective problem, the multi-objective reformulation introduces an auxiliary objective function, for which a multi-objective EA is then used as the optimizer. Scharnow et al. (2002) suggested that multi-objective reformulation could be superior to the original single objective formulation. This was confirmed for various problems [Neumann and Wegener, 2005; 2007; Friedrich et al., 2010; Neumann et al., 2008] by showing that while a single-objective EA gets stuck, multi-objective reformulation solves the problems efficiently. Regarding approximations, it has been shown that multi-objective EAs are effective for some NP-hard problems, including the minimum set cover problem[Friedrich et al., 2010] and the minimum multicuts problem [Neumann and Reichel, 2008].

In the present study, we investigate the approximation performance of EAs by introducing a framework simple evolutionary algorithm with isolated population (SEIP), which can be implemented as a single- or multi-objective EA. We obtain general approximation results of SEIP; and then on the minimum set cover problem (MSCP), we prove SEIP achieves the currently best-achievable result, i.e., for the unbounded MSCP, SEIP efficiently obtains an Hk-approximation ratio (where Hk is the harmonic number of the cardinality of the largest set), the asymptotic lower bound [Feige, 1998], and for the minimum k-set cover problem, SEIP efficiently yields an (Hkk-1 8k9 )-approximation ratio, the currently bestachievable quality [Hassin and Levin, 2005]. Moreover, on a subclass of the problem, we demonstrate how SEIP can overcome the difficulty that limits the greedy algorithm.

In the remainder of the paper, Section 2 introduces SEIP, Section 3 presents the main results, and Section 4 concludes.

2 SEIP

We use bold small letters such as w, x, y, z to represent vectors. Denote [m] as the set {1, 2, . . . ,m} and 2S as the power set of S, which consists of all subsets of S. Denote Hn = ∑ni=1 1i for the nth harmonic number. Note that Hn ∼ lnn since lnn ≤ Hn ≤ lnn + 1. We consider minimization problems as follows.

Definition 1 (Minimization problem)

Given an evaluation function f and a set of feasibility constraints C, find a solution x ∈ {0, 1}n that minimizes f(x) while satisfying constraints in C. A problem instance can be specified by its parameters (n, f, C).

A solution is called feasible if it satisfies all the constraints in C; otherwise, it is called an infeasible solution, or a partial solution in this paper. Here, we assume that the evaluation function f is defined on the full input space, that is, it evaluates all possible solutions regardless of their feasibility.

The SEIP framework is depicted in Algorithm 1. It uses an isolation function μ to isolate solutions. For some integer q, the function μ maps a solution to a subset of [q]. If and only if two solutions x1 and x2 are mapped to subsets with the same cardinality, that is, |μ(x1)| = |μ(x2)|, the two solutions compete with each other. In that case, we say the two solutions are in the same isolation, and there are at most q+1 isolations since the subsets of [q] have q + 1 different cardinalities.

Algorithm 1 (SEIP)

Given a minimization problem (n, f, C), an isolation function μ encodes each solution as a binary vector of length m. SEIP minimization of f with respect to constraint C consists of the following steps.

1: P ← {x∅ = (0, 0, . . . , 0)}

• 2: while stop ̸= true do
• 3: Choose x ∈ P uniformly at random
• 4: Mutate x to generate x′ 5: if ∀x ∈ P : superior(x, x′ ) = false then 6: Q← {x ∈ P | superior(x′ , x) = true} 7: P ←P ∪ {x′ }-Q 8: end if 9: end while
• 10: return the best feasible solution in P

where the superior function of two solutions determines whether one solution is superior to the other. This is defined as follows: superior(x, y) is true if both of the following rules are satisfied:

The mutation step commonly employs two operators: One-bit mutation: Flip one randomly selected bit position of x to generate x′ .

Bit-wise mutation: Flip each bit of x with probability 1m to generate x′ .

Usually, one-bit mutation is considered for local searches, while bit-wise mutation is suitable for global searches as it has a positive probability for producing any solution. We denote SEIP with one-bit mutation as LSEIP ("L" for "local") and with bit-wise mutation as GSEIP ("G" for "global"). The stop criterion is not described in the definition of SEIP, since EAs are usually used as anytime algorithms. In our analysis we will determine the approximation performance as well as the corresponding computational complexity.

By configuring the isolation function, SEIP can be implemented as a single- or multi-objective EA. When the isolation function puts all solutions in one isolation, all solutions competes with each other and thus SEIP will equal the single objective (1+1)-EA. The isolation function is also be configured to simulate the dominance relationship of multiobjective EAs such as SEMO/GSEMO [Laumanns et al., 2002] and DEMO [Neumann and Reichel, 2008]. If we are dealing with k-objective optimization with discrete objective values, a simple approach is to use one of the objective functions, say f1, as the fitness function and use the combination of the values of the remaining k - 1 objective functions (say f2, . . . , fk) as the isolation functions. In such way, all the non-dominant solutions of a multi-objective EA are also kept in the SEIP population. Hence, SEIP can be viewed as a generalization of multi-objective EAs in terms of the solutions retained.

3 Approximation Performance of SEIP

3.1 General Approximation Results

Given a minimization problem, we denote xOPT as an optimal solution of the problem, and OPT = f(xOPT ). For a feasible solution x, we regard the ratio f(x) OPT as its approximation ratio. If the approximation ratio of a feasible solution is upper-bounded by some value r, that is, 1 ≤ f(x) OPT ≤ r, the solution is called an r-approximate solution. An algorithm that guarantees to find an r-approximate solution for an arbitrary problem instance in polynomial time is an rapproximation algorithm. We analyze the approximation ratio obtained by SEIP.

For minimization problems, we consider linearly additive isolation functions. μ is a linearly additive isolation function if, for some integer q,

The quality of a feasible solution is measured in terms of the approximation ratio. To measure the quality of a partial (infeasible) solution, we define a partial reference function and partial ratio as follows.

Definition 2 (Partial reference function)

Given a set [q] and a value v, a function L[q],v : 2[q] →R is a partial reference function if L[q],v([q]) = v, and L[q],v(R1) = L[q],v(R2) for all R1,R2 ⊆ [q] such that |R1| = |R2|.

For a minimization problem with optimal cost OPT and an isolation function μ mapping feasible solutions to the set [q], we denote a partial reference function with respect to [q] and OPT as L[q],OPT . When the problem and the isolation function are clear, we omit the subscripts and simply denote the partial reference function as L.

We then can define the partial ratio of a solution x as p-ratio(x) = f(x) L(μ(x)) and the conditional partial ratio of y conditioned on x as p-ratio(x | y) = f(y|x) L(μ(y)|μ(x)) , under the given minimization problem (n, f, C), the isolation function μ, and the partial reference function L, where f(y | x) = f(x ∪ y) - f(x) and L(μ(y) | μ(x)) = L(μ(y) ∪ μ(x)) -L(μ(x)). The partial ratio is an extension of the approximation ratio, noticing that the partial ratio for a feasible solution equals its approximation ratio. We have two properties of the partial ratio: one is that it is non-increasing in SEIP, and the other is its decomposability. As the consequence, we can show the following theorem.

Theorem 1

Given a minimization problem (n, f, C) and an isolation function μ mapping to subsets of [q], assume that every solution is encoded in an m-length binary vector. For some constant r ≥ 1 with respect to a corresponding partial reference function L, if

1. p-ratio(x∅ ) ≤ r,
2. for every partial solution x such that p-ratio(x) ≤ r, SEIP takes x as the parent solution and generates an offspring partial solution y such that μ(x) ⊂ μ(x ∪ y) and p-ratio(y | x) ≤ r in polynomial time in q and m,

then SEIP finds an r-approximate solution in polynomial time in q and m.

Theorem 1 reveals how SEIP can work to achieve an approximate solution. Starting from the empty set, SEIP uses its mutation operator to generate solutions in all isolations, and finally generates a feasible solution. During the process, SEIP repeatedly optimizes the partial ratio in each isolation by finding partial solutions with better conditional partial ratios. Since the feasible solution can be viewed as a composition of a sequence of partial solutions from the empty set, the approximation ratio is related to the conditional partial ratio of each partial solution. Moreover, for GSEIP, we can further have a tighter approximation ratio by Theorem 2.

Definition 3 (Non-negligible path)

Given a minimization problem (n, f, C) and an isolation function μ mapping to subsets of [q], assume that every solution is encoded in an m-length binary vector. A set of solutions N is a non-negligible path with ratios {ri}q-1 i=0 and gap c if x∅ ∈ N and, for every solution x ∈ N, there exists a solution x′ = (x ∪ y+ - y-) ∈ N, where the pair of solutions (y+, y-) satisfies

Theorem 2

Given a minimization problem (n, f, C) and an isolation function μ mapping to subsets of [q], assume that every solution is encoded in an m-length binary vector. If there exists a nonnegligible path with ratios {ri}q-1 i=0 and gap c, then GSEIP finds an (∑q-1 i=0 ri)-approximate solution in expected time O(q2mc ).

Using Theorem 2 to prove the approximation ratio of GSEIP for a specific problem, we need to find a nonnegligible path and then calculate the conditional evaluation function for each jump on the path. One way of finding a non-negligible path for a problem is to follow an existing algorithm for the problem. This will lead to a proof that the EA can achieve the same approximation ratio by simulating the existing algorithm. Similar ideas have been used to confirm that EAs can simulate dynamic programming [Doerr et al., 2009]. In addition, note that the concrete form of the partial reference function is not required in the theorem.

3.2 Approximation Results onMSCP

In the definition of the minimization problem, solutions are represented in binary vectors. When the aim of a minimization problem is to find a subset from a universal set, we can equivalently use a binary vector to represent a subset, where each element of the vector indicates the membership of a corresponding element of the universe set. We denote x∅ = (0, . . . , 0) as the vector corresponding to the empty set.

We investigate the minimum set cover problem (MSCP), which is an NP-hard problem.

Definition 4 (Minimum set cover problem (MSCP))

Given a set of n elements U = [n] and a collection C = {S1, S2, . . . , Sm} of m nonempty subsets of U, where each Si is associated with a positive cost w(Si), find a subset X∗ ⊆ C such that ∑S∈X∗ w(S) is minimized with respect to ∪ S∈X∗ S = U.

Definition 5 (Minimum k-set cover problem)

An MSCP (n,w, C, U) is a k-set cover problem if, for some constant k, it holds that |S| ≤ k for all S ∈ C, denoted as (n, w, C, U, k).

To apply SEIP to the MSCP, we use the fitness function f(x) = w · x, which has the objective of minimizing the total weight. For a solution x, we denote R(x) = ∪S∈x(C)S, that is, R(x) is the set of elements covered by x. We use the isolation function μ(x) = R(x), which makes two solutions compete only when they cover the same number of elements owing to the effect of the isolation function. We could regard a partial reference function L of x to be the minimum price that optimal solutions pay for covering the same number of elements covered by x, although it is not necessary to calculate the partial reference function.

For the (unbounded) MSCP, we show that SEIP achieves an Hk-approximation ratio in Theorem 3, where k is the cardinality of the largest set.

Theorem 3

Given an MSCP (n,w,C,U) where |C| = m, LSEIP and GSEIP find an Hk-approximate solution in expected time O(mn2 ), where k is the size of the largest set in C.

The theorem is proved via the construction of a nonnegligible path following the greedy algorithm [Chv´atal, 1979]. It has been proven that a multi-objective EA named SEMO [Friedrich et al., 2010] achieves the same approximation ratio, which is known as the asymptotic lower bound for the problem. The theorem confirms that SEIP can simulate multi-objective EAs in terms of the solution retained, so that SEIP is able to achieve the approximation ratio obtained by multi-objective EAs.

For the minimum k-set cover problem, we show that GSEIP achieves the same (Hk - k-1 8k9 )-approximation ratio as the GAWW algorithm [Hassin and Levin, 2005] in Theorem 4, which is the current best-achievable result.

Theorem 4

Given a minimum k-set cover problem (n, w, C, U, k), where |C| = m, we denote R(x) = ∪S∈xS. GSEIP using μ(x) = R(x) finds an (Hk - k-1 8k9 )-approximate solution in expected time O(mk+1n2 ).

The theorem is proved via the construction of a nonnegligible path following the GAWW algorithm. This result reveals that when the problem is bounded, which is very likely in real-world situations, GSEIP can yield a better approximation ratio than in the unbounded situation. Since the greedy algorithm cannot achieve an approximation ratio lower than Hn, the result also implies that GSEIP has essential non-greedy behaviors for approximations.

For a comparison of the greedy algorithm, LSEIP and GSEIP, we study on a subclass of the minimum k-set cover problem, denoted as problem I. Problem I is a minimum kset cover problem (n,w,C,U) constructed as follows. Note that n = kL for some integer L. The optimal solution consists of L non-overlapping sets {S∗ i }Li=1 , each of which contains k elements in U. Imagine that elements in each S∗ i are ordered and let Sij contain only the jth element of set S∗ i . The collection of sets C consists of all sets of S∗ i and Sij. Assign each S∗ i weight 1+ϵ for some ϵ > 0, and assign each Sij weight 1/j. Figure 1 illustrates Problem I.

Proposition 1

Given an arbitrary value ξ > 0, the approximation ratio of the greedy algorithm for problem I is lower-bounded by Hk - ξ.

Proposition 2

With probability of at least 1 - 1 k+1 , LSEIP finds a (Hk - kn (Hk -1))-approximate solution for problem I in O ((1 + 1k )n3 ) expected steps.

Proposition 3

GSEIP finds the optimal solution for problem I in O( 1k · (1 + 1k )k+1 · nk+3 ) expected steps.

For GSEIP, we can also derive Proposition 4 that discloses the time-dependent approximation performance of GSEIP, which illustrates an interesting property: compared with the greedy algorithm, whose performance cannot be improved given extra time, SEIP always seeks better approximate solutions. Users may allocate more running time in a trade-off for better solutions.

Proposition 4

GSEIP finds an (Hk - c kn (Hk -1))-approximate solution for problem I in O(mn2 + cn2mk+1 /k) expected steps, for c = 0, 1, . . . , nk .

4 Conclusion

We studied the approximation performance of an EA framework, SEIP, introducing an isolation function to manage competition among solutions, which can be configured as a single- or multi-objective EA. We analyzed the approximation performance of SEIP using the partial ratio and obtained a general characterization of the SEIP approximation behavior. Our analysis discloses that SEIP can achieve a guaranteed approximate solution: it tries to optimize the partial ratio of solutions in every isolation by finding good partial solutions, then these partial solutions form a feasible solution with a good approximation ratio. We then took the MSCP as a benchmark problem. Our analysis shows that for the unbounded MSCP, SEIP achieves an Hk-approximation ratio (where Hk is the harmonic number of the cardinality of the largest set), the asymptotic lower bound. For the minimum kset cover problem, it achieves an (Hk - k-1 8k9 )-approximation ratio, the current best-achievable result [Hassin and Levin, 2005], which is beyond the ability of the greedy algorithm. Moreover, for a subclass of the MSCP, we show how SEIP with either one-bit or bit-wise mutation can overcome the difficulty that limits the greedy algorithm.

We have shown that SEIP can simulate the greedy algorithm. Since the greedy algorithm is a general scheme for approximations and has been analyzed for many problems, SEIP analysis can be extended to cases for which the greedy algorithm has been applied, and therefore we can easily show that SEIP is a 1k -approximation algorithm for k-extensible systems [Mestre, 2006], including b-matching, maximum profit scheduling and maximum asymmetric TSP problems. Moreover, to prove the approximation ratio of SEIP for the minimum k-set cover problem, we used SEIP to simulate the GAWW algorithm, which implies that SEIP also has extra behaviors that provide opportunities to exceed the greedy algorithm. Limitations of SEIP can also be found from Theorem 2. When SEIP is required to flip a number of bits at a time, which depends on n, to achieve a good solution, we will find that the one-bit mutation is limited since it only flips one bit at a time, and the bit-wise mutation is also limited as it requires exponential time.

Our theoretical analysis suggests that EAs can achieve solutions with guaranteed performance. We believe that guaranteed and better approximation performance can be achieved by better EA design in the future.

Acknowledgements

We thank Dr. Per Kristian Lehre and Dr. Yitong Yin for helpful discussions. This work was partly supported by the National Fundamental Research Program of China (2010CB327903), the National Science Foundation of China (61073097, 61021062), EPSRC (UK) (EP/I010297/1), and an EU FP7-PEOPLE-2009-IRSES project under Nature Inspired Computation and its Applications (NICaiA) (247619).