The FastMap Algorithm for Shortest Path Computations

1 Introduction and RelatedWork

Shortest path computations commonly occur in the inner procedures of many AI programs. In video games, for example, a large fraction of CPU cycles is spent on shortest path computations [Uras and Koenig, 2015]. Many other tasks in AI, including motion planning, temporal reasoning, and decision making [Russell and Norvig, 2009], also involve finding and reasoning about shortest paths. While Dijkstra's algorithm [Dijkstra, 1959] can be used to compute shortest paths in polynomial time, speeding up shortest path computations allows one to solve the aformentioned tasks faster. One way to do that is to use A* search with an informed heuristic [Hart et al., 1968].

A perfect heuristic is one that returns the true distance between any two nodes in a given graph. A* with such a heuristic and proper tie-breaking is guaranteed to expand nodes only on a shortest path between the given start and goal nodes. In general, computing the perfect heuristic between two nodes is as hard as computing the shortest path between them. Hence, A* search benefits from a perfect heuristic only if it is computed offline. However, precomputing all pairwise distances is not only time-intensive but also requires a prohibitive O(N 2 ) memory where N is the number of nodes. The memory requirements for storing all-pairs shortest paths data can be somewhat addressed through compression [Botea and Harabor, 2013; Strasser et al., 2015].

Existing methods for preprocessing a given graph (without precomputing all pairwise distances) can be grouped into the following categories: Hierarchical abstractions that yield suboptimal paths have been used to reduce the size of the search space by abstracting groups of nodes [Botea et al., 2004; Sturtevant and Buro, 2005]. More informed heuristics [Bj¨ornsson and Halld´orsson, 2006; Cazenave, 2006; Sturtevant et al., 2009] focus A* searches better, resulting in fewer expanded nodes. Hierarchies can also be used to derive heuristics during the search [Leighton et al., 2008; Holte et al., 1994]. Dead-end detection and other pruning methods [Bj¨ornsson and Halld´orsson, 2006; Goldenberg et al., 2010; Pochter et al., 2010] identify areas of the graph that do not need to be searched to find shortest paths. Search with contraction hierarchies [Geisberger et al., 2008] is an optimal hierarchical method, where every level of the hierarchy contains only a single node. It has been shown to be efficient on road networks but seems to be less efficient on graphs with higher branching factors, such as gridbased game maps [Storandt, 2013]. N-level graphs [Uras and Koenig, 2014], constructed from undirected graphs by partitioning the nodes into levels, also allow for significant pruning during the search.

A different approach that does not rely on preprocessing of the graph is to use some notion of a geometric distance between two nodes as a heuristic of the distance between them. One such heuristic for gridworlds is the Manhattan Distance heuristic.1 For many gridworlds, A* search using the Manhattan Distance heuristic outperforms Dijkstra's algorithm. However, in complicated 2D/3D gridworlds like mazes, the Manhattan Distance heuristic may not be sufficiently informed to focus A* searches effectively. Another issue associated with Manhattan Distance-like heuristics is that they are not well defined for general graphs.2 For a graph that cannot be conceived in a geometric space, there is no closedform formula for a "geometric" heuristic for the distance between two nodes because there are no coordinates associated with them.

1 In a 4-neighbor 2D gridworld, for example, the Manhattan Distance between two cells (x1, y1) and (x2, y2) is |x1-x2|+|y1-y2|. Generalizations exist for 8-neighbor 2D and 3D gridworlds.

For a graph that does not already have a geometric embedding in Euclidean space, a preprocessing algorithm can be used to generate one. As described before, at runtime, A* search would then use the Euclidean distance between any two nodes in this space as an estimate for the distance between them in the given graph. One such approach is Euclidean Heuristic Optimization (EHO) [Rayner et al., 2011]. EHO guarantees admissiblility and consistency and therefore generates shortest paths. However, it requires solving a Semidefinite Program (SDP). SDPs can be solved in polynomial time [Vandenberghe and Boyd, 1996]. EHO leverages additional structure to solve them in cubic time. Still, a cubic preprocessing time limits the size of the graphs that are amenable to this approach.

The Differential heuristic is another state-of-the-art approach that has the benefit of a near-linear runtime. However, unlike the approach in [Rayner et al., 2011], it does not produce an explicit Euclidean embedding. In the preprocessing phase of the Differential heuristic approach, some nodes of the graph are chosen as pivot nodes. The distances between each pivot node and every other node are precomputed and stored [Sturtevant et al., 2009]. At runtime, the heuristic between two nodes a and b is given by maxp |d(a, p) -d(p, b)|, where p is a pivot node and d(·, ·) is the precomputed distance. The preprocessing time is linear in the number of pivots times the size of the graph. The required space is linear in the number of pivots times the number of nodes, although a more succinct representation is presented in [Goldenberg et al., 2011]. Similar preprocessing techniques are used in Portal-Based True Distance heuristics [Goldenberg et al., 2010].

In this paper, we present a new preprocessing algorithm, called FastMap, that produces an explicit Euclidean embedding while running in near-linear time. It therefore has the benefits of the small preprocessing time of the Differential heuristic approach and of producing an embedding from which a heuristic between two nodes can be quickly computed using a closed-form formula. Our preprocessing algorithm, dubbed FastMap, is inspired by the data-mining algorithm of the same name [Faloutsos and Lin, 1995]. It is orders of magnitude faster than SDP-based approaches for producing Euclidean embeddings. FastMap also produces admissible and consistent heuristics and therefore guarantees the generation of shortest paths.

The FastMap heuristic has several advantages: First, it is defined for general (undirected) graphs. Second, we observe empirically that, in gridworlds, A* using the FastMap heuristic runs faster than A* using the Manhattan or Octile distance heuristics. A* using the FastMap heuristic runs equally fast or faster than A* using the Differential heuristic, with the same memory resources. The (explicit) Euclidean embedding produced by FastMap also has representational benefits like recovering the underlying manifolds of the graph and/or visualizing them. Moreover, we observe that the FastMap and Differential heuristics have complementary strengths and can be easily combined to generate an even more informed heuristic.

2Henceforth, whenever we refer to a graph, we mean an edgeweighted undirected graph unless stated otherwise.

2 The Origin of FastMap

The FastMap algorithm [Faloutsos and Lin, 1995] was introduced in the data-mining community for automatically generating geometric embeddings of abstract objects. For example, if we are given objects in form of long DNA strings, multimedia datasets such as voice excerpts or images or medical datasets such as ECGs or MRIs, there is no geometric space in which these objects can be naturally visualized. However, there is often a well defined distance function between every pair of objects. For example, the edit distance3 between two DNA strings is well defined although an individual DNA string cannot be conceptualized in geometric space.

Clustering techniques, such as the k-means algorithm, are well studied in machine learning [Alpaydin, 2010] but cannot be applied directly to domains with abstract objects because they assume that objects are described as points in geometric space. FastMap revives their applicability by first creating a Euclidean embedding for the abstract objects that approximately preserves the pairwise distances between them. Such an embedding also helps to visualize the abstract objects, for example, to aid physicians in identifying correlations between symptoms frommedical records.

The data-mining FastMap gets as input a complete undirected edge-weighted graph G = (V, E). Each node vi ∈ V represents an abstract object Oi. Between any two nodes vi and vj there is an edge (vi, vj) ∈ E with weight D(Oi,Oj) that corresponds to the given distance between objects Oi and Oj. A Euclidean embedding assigns to each object Oi a Kdimensional point pi ∈ RK . A good Euclidean embedding is one in which the Euclidean distance between any two points pi and pj closely approximates D(Oi,Oj).

One of the early approaches for generating such an embedding is based on the idea of multi-dimensional scaling (MDS) [Torgerson, 1952]. Here, the overall distortion of the pairwise distances is measured in terms of the "energy" stored in "springs" that connect each pair of objects. MDS, however, requires O(|V |2 ) time and hence does not scale well in practice. On the other hand, FastMap [Faloutsos and Lin, 1995] requires only linear time. Both methods embed the objects in a K-dimensional space for a user-specified K.

FastMap works as follows: In the very first iteration, it heuristically identifies the farthest pair of objects Oa and Ob in linear time. It does this by initially choosing a random object Ob and then choosing Oa to be the farthest object away from Ob. It then reassigns Ob to be the farthest object away from Oa. Once Oa and Ob are determined, every other object Oi defines a triangle with sides of lengths dai = D(Oa,Oi), dab = D(Oa,Ob) and dib = D(Oi,Ob). Figure 1(a) shows this triangle. The sides of the triangle define its entire geometry, and the projection of Oi onto OaOb is given by xi = (d2ai + d2ab - d2ib )/(2dab). FastMap sets the first coordinate of pi, the embedding of object Oi, to xi. In particular, the first coordinate of pa is xa = 0 and of pb is xb = dab. Computing the first coordinates of all objects takes only linear time since the distance between any two objects Oi and Oj for i, j /∈ {a, b} is never computed.

3The edit distance between two strings is the minimum number of insertions, deletions or substitutions that are needed to transform one to the other.

In the subsequent K - 1 iterations, the same procedure is followed for computing the remaining K - 1 coordinates of each object. However, the distance function is adapted for different iterations. For example, for the first iteration, the coordinates of Oa and Ob are 0 and dab, respectively. Because these coordinates fully explain the true distance between them, from the second iteration onwards, the rest of pa and pb's coordinates should be identical. Intuitively, this means that the second iteration should mimic the first one on a hyperplane that is perpendicular to the line OaOb. Figure 1(b) explains this intuition. Although the hyperplane is never constructed explicitly, its conceptualization implies that the distance function for the second iteration should be changed in the following way: Dnew(O′i , O′j )2 = D(Oi, Oj )2 -(xi - xj)2 . Here, O′i and O′j are the projections of Oi and Oj, respectively, onto this hyperplane, and Dnew is the new distance function.

3 FastMap for Shortest Path Computations

In this section, we provide the high-level ideas for how to adapt the data-mining FastMap algorithm to shortest path computations. In the shortest path computation problem, we are given a non-negative edge-weighted undirected graph G = (V,E) along with a start node vs and a goal node vg. As a preprocessing technique, we can embed the nodes of G in a Euclidean space. As A* searches for a shortest path from vs to vg, it can use the Euclidean distance from v ∈ V to vg as a heuristic for v. The number of node expansions of A*

search depends on the informedness of the heuristic which, in turn, depends on the ability of the embedding to preserve the pairwise distances.

The idea is to view the nodes of G as the objects to be embedded in Euclidean space. As such, the data-mining FastMap algorithm cannot directly be used for generating an embedding in linear time. The data-mining FastMap algorithm assumes that the distance dij between two objects Oi and Oj can be computed in constant time, independent of the number of objects. Computing the distance between two nodes depends on the size of the graph. Another problem is that the Euclidean distances may not satisfy important properties such as admissibility or consistency. Admissibility guarantees that A* finds shortest paths while consistency allows A* to avoid re-expansions of nodes as well.

The first issue of having to retain (near-)linear time complexity can be addressed as follows: In each iteration, after we identify the farthest pair of nodes Oa and Ob, the distances dai and dib need to be computed for all other nodes Oi. Computing dai and dib for any single node Oi can no longer be done in constant time but requires O(|E| + |V | log |V |) time instead [Fredman and Tarjan, 1984]. However, since we need to compute these distances for all nodes, computing two shortest path trees rooted at nodes Oa and Ob yields all necessary distances. The complexity of doing so is also O(|E| + |V | log |V |), which is only linear in the size of the graph.4 The amortized complexity for computing dai and dib for any single node Oi is therefore near-constant time.

The second issue of having to generate a consistent (and thus admissible) heuristic is formally addressed in Theorem 1. The idea is to use L1 distances instead of L2 distances in each iteration of FastMap. The mathematical properties of the L1 distance can be used to prove that admissibility and consistency hold irrespective of the dimensionality of the embedding.

Algorithm 1 presents data-mining FastMap adapted to the shortest path computation problem. The input is an edgeweighted undirected graph G = (V,E,w) along with two user-specified parameters Kmax and ϵ. Kmax is the maximum number of dimensions allowed in the Euclidean embedding. It bounds the amount of memory needed to store the Euclidean embedding of any node. ϵ is the threshold that marks a point of diminishing returns when the distance between the farthest pair of nodes becomes negligible. The output is an embedding pi ∈ RK (with K ≤ Kmax) for each node vi ∈ V .

The algorithm maintains a working graph G′ = (V, E, w′ ) initialized to G. The nodes and edges of G′ are always identical to those of G but the weights on the edges of G′ change with every iteration. In each iteration, the farthest pair (na, nb) of nodes in G′ is heuristically identified in near-linear time (line 4). The Kth coordinate [pi]K of each node vi is computed using a formula similar to that for xi in Figure 1(a). However, that formula is modified to (dai + dab - dib)/2 to ensure admissibility and consistency of the heuristic. In each iteration, the weight of each edge ALGORITHM 1: Shows the FastMap algorithm. G = (V, E, w) is a nonnegative edge-weighted undirected graph; Kmax is the user-specified upper bound on the dimensionality; ϵ is a user-specified threshold; K ≤ Kmax is the dimensionality of the computed embedding; pi is the Euclidean embedding of node vi ∈ V . Line 11 is equivalent to w′ (u, v) = w(u, v) - ‖pu - pv‖1.

4 unless |E| = O(|V |), in which case the complexity is nearlinear in the size of the input because of the log |V | factor

Input: G = (V, E, w), Kmax and ϵ. Output: K and pi ∈ RK for all vi ∈ V . 1 w′ = w; K = 1; 2 while Kmax > 0 do 3 Let G′ = (V,E,w′ ); 4 (na, nb) ← GetFarthestPair(G′ ); 5 Compute shortest path trees rooted at na and nb on G′ to obtain dab, dai and dib for all vi ∈ V ; 6 if dab < ϵ then 7 Break; 8 for each v ∈ V do 9 [pv]K = (dav + dab - dvb)/2 10 for each edge (u, v) ∈ E do 11 w′ (u, v) = w′ (u, v) - |[pu]K - [pv]K|; 12 K =K +1; Kmax =Kmax -1;

is decremented to resemble the update rule for Dnew in Figure 1(b) (line 11). However, that update rule is modified to w′ (u, v) = w′ (u, v)-|[pu]K-[pv]K| to use the L1 distances instead of the L2 distances.

The method GetFarthestPair(G′ ) (line 4) computes shortest path trees in G′ a small constant number of times, denoted by τ.5 It therefore runs in near-linear time. In the first iteration, we assign na to be a random node. A shortest path tree rooted at na is computed to identify the farthest node from it. nb is assigned to be this farthest node. In the next iteration, a shortest path tree rooted at nb is computed to identify the farthest node from it. na is reassigned to be this farthest node. Subsequent iterations follow the same switching rule for na and nb. The final assignments of nodes to na and nb are returned after τ iterations. This entire process of starting from a randomly chosen node can be repeated a small constant number of times.6

Figure 2 shows the working of our algorithm on a small gridworld example.

3.1 Proof of Consistency

In this section, we prove the consistency of the FastMap heuristic. Since consistency implies admissibility, this also proves that A* with the FastMap heuristic returns shortest paths. We use the following notation in the proofs: wixy is the weight on the edge between nodes x and y in the ith iteration; dixy is the distance between nodes x and y in the ith iteration (using the weights wi ); px is the vector of coordinates produced for node x, and [px]j is its j th coordinate;7

5 τ = 10 in our experiments.

6This constant is also 10 in our experiments.

7The ith iteration sets the value of [px]i.

hixy is the FastMap heuristic between nodes x and y after i iterations. Note that hixy is the L1 distance between px and py at iteration i, that is hixy := ∑ij=1 |[px]j - [py]j|. We also define ∆i+1 xy := dixy - di+1 xy . In the following proofs, we use the fact that |A| + |B| ≥ |A+B| and |A| - |B| ≤ |A-B|.

Proof. We prove by induction that, in any iteration i, wiuv ≥ 0 for all (u, v) ∈ E. Thus, the weight of each edge in the ith iteration is non-negative and therefore diuv ≥ 0 for all u, v. For the base case, w1uv = w(u, v) ≥ 0. We assume that wiuv ≥ 0 and show that wi+1 uv ≥ 0. Let na and nb be the farthest pair of nodes identified in the ith iteration. From lines 9 and 11, wi+1 uv = wiuv -|(diau -diav )/2+(divb -diub )/2|. To show that wi+1 uv ≥ 0 we show that wiuv ≥ |(diau - diav )/2 + (divb - diub )/2|. From the triangle inequality, for any node l, diuv +min(diul , dilv ) ≥ max(diul , dilv ). Therefore, diuv ≥ |diul -dilv | and thus also 2diuv ≥ |diua -diav |+|diub -dibv |. This means that diuv ≥ |diau - diav |/2 + |divb - diub |/2. Therefore, diuv ≥ |(diau - diav )/2 + (divb - diub )/2|. This concludes the proof since wiuv ≥ diuv .

Proof. Let 〈u1 = x, . . . , um = y〉 be the shortest path from x to y in iteration i. By definition, dixy = ∑m-1 j=1 wiujuj+1 and di+1 xy ≤ ∑m-1 j=1 wi+1 ujuj+1 . From line 11, wi+1 ujuj+1 = wiujuj+1 - |[puj ]i - [puj+1 ]i|. Therefore, ∆i+1 xy = dixy - di+1 xy ≥ ∑m-1 j=1 |[puj ]i - [puj+1 ]i|. This concludes the proof since ∑m-1 j=1 |[puj ]i-[puj+1 ]i| ≥ | ∑m-1 j=1 [puj ]i-[puj+1 ]i| = |[px]i - [py]i|.

Proof. We prove the lemma by induction on i. The base case for i = 1 is implied by Lemma 2. We assume that d1xy +hiyg - hixg ≥ di+1 xy and show d1xy +hi+1 yg -hi+1 xg ≥ di+2 xy . We know that hi+1 yg -hi+1 xg = hiyg-hixg-(|[px]i+1-[pg]i+1|-|[py]i+1[pg]i+1|). Since |[px]i+1 - [pg]i+1| - |[py]i+1 - [pg]i+1| ≤ |[px]i+1 - [py]i+1|, we have hi+1 yg - hi+1 xg ≥ hiyg - hixg - |[px]i+1-[py]i+1|. Hence, d1xy +hi+1 yg -hi+1 xg ≥ (d1xy +hiyg - hixg ) - |[px]i+1 - [py]i+1|. Using the inductive assumption, we get d1xy + hi+1 yg - hi+1 xg ≥ di+1 xy - |[px]i+1 - [py]i+1|. By definition, di+1 xy = ∆i+2 xy + di+2 xy . Substituting for di+1 xy , we get d1xy + hi+1 yg - hi+1 xg ≥ di+2 xy + (∆i+2 xy - |[px]i+1 - [py]i+1|). Lemma 2 shows that ∆i+2 xy ≥ |[px]i+1 - [py]i+1|, which concludes the proof.

Theorem 1. The FastMap heuristic is consistent.

Proof. For all x, y, g and i: From Lemma 3, we know d1xy + hiyg - hixg ≥ di+1 xy . From Lemma 1, we know di+1 xy ≥ 0. Put together, we have d1xy + hiyg ≥ hixg . Finally, higg = ∑ij=1 |[pg]j - [pg]j | = 0.

Theorem 2. The informedness of the FastMap heuristic increases monotonically with the number of dimensions.

Proof. This theorem follows from the fact that for any two nodes x and g, hi+1 xg = hixg + |[px]i+1 - [pg]i+1| ≥ hixg .

4 Experimental Results

We performed experiments on many benchmark maps from [Sturtevant, 2012]. Figure 3 presents representative results. The FastMap heuristic (FM) and the Differential heuristic (DH) with equal memory resources8 are compared against each other. In addition, we include the Octile heuristic (OCT) as a baseline, that also uses a closed-form formula for the computation of its heuristic.

We observe that, as the number of dimensions increases, (a) FM and DH perform better than OCT; (b) the median number of expanded nodes when using the FM heuristic decreases (which is consistent with Theorem 2); and (c) the median absolute deviation (MAD) of the number of expanded nodes when using the FM heuristic decreases. When FM's MADs are high, the variabilities can possibly be exploited in future work using Rapid Randomized Restart strategies.

FastMap also gives us a framework for identifying a point of diminishing returns with increasing dimensionality. This happens when the distance between the farthest pair of nodes stops being "significant". For example, such a point is observed in Figure 3(f) around dimensionality 5.9

In mazes, such as in Figure 3(g), A* using the DH heuristic outperforms A* using the FM heuristic. This leads us to believe that FM provides good heuristic guidance in domains that can be approximated with a low-dimensional manifold. This observation also motivates us to create a hybrid FM+DH heuristic by taking the maximum of the two heuristics. Some relevant results are shown in Table 1. We use FM(K) to denote the FM heuristic with K dimensions and DM(K) to denote the DH heuristic with K pivots. For the results in Table 1, all heuristics have equal memory resources. We observe that the number of node expansions of A* using the FM(5)+DH(5) heuristic is always second best compared to A* using the FM(10) heuristic and A* using the DH(10) heuristic. On one hand, this decreases the percentages of instances on which it expands the least number of nodes (as seen in the second row of Table 1). But, on the other hand, its median number of node expansions is not far from that of the best technique in each breakdown.

8The dimensionality of the Euclidean embedding for FM matches the number of pivots in DH.

9The distances between the farthest pair of nodes, computed on line 4 of Algorithm 1, for the first 10 dimensions are: 〈581, 36, 22, 15, 14, 10, 6, 6, 5, 4〉.

5 Conclusions

We presented a near-linear time preprocessing algorithm, called FastMap, for producing a Euclidean embedding of a general edge-weighted undirected graph. At runtime, the Euclidean distances were used as heuristic by A* for shortest path computations. We proved that the FastMap heuristic is admissible and consistent, thereby generating shortest paths. FastMap produces the Euclidean embedding in nearlinear time, which is significantly faster than competing approaches for producing Euclidean embeddings with optimality guarantees that run in cubic time. We also showed that it is competitive with other state-of-the-art heuristics derived in near-linear preprocessing time. However, FastMap has the combined benefits of requiring only near-linear preprocessing time and producing explicit Euclidean embeddings that try to recover the underlying manifolds of the given graphs.

Acknowledgments

This work was supported by NSF under grant numbers 1724392, 1409987, and 1319966.