Efficient Single-Source Shortest Path and Distance Queries on Large Graphs (1306.1153v1)

Abstract: This paper investigates two types of graph queries: {\em single source distance (SSD)} queries and {\em single source shortest path (SSSP)} queries. Given a node $v$ in a graph $G$, an SSD query from $v$ asks for the distance from $v$ to any other node in $G$, while an SSSP query retrieves the shortest path from $v$ to any other node. These two types of queries are fundamental building blocks of numerous graph algorithms, and they find important applications in graph analysis, especially in the computation of graph measures. Most of the existing solutions for SSD and SSSP queries, however, require that the input graph fits in the main memory, which renders them inapplicable for the massive disk-resident graphs commonly used in web and social applications. The only exceptions are a few techniques that are designed to be I/O efficient, but they all focus on undirected and/or unweighted graphs, and they only offer sub-optimal query efficiency. To address the deficiency of existing work, this paper presents {\em Highways-on-Disk (HoD)}, a disk-based index that supports both SSD and SSSP queries on directed and weighted graphs. The key idea of HoD is to augment the input graph with a set of auxiliary edges, and exploit them during query processing to reduce I/O and computation costs. We experimentally evaluate HoD on both directed and undirected real-world graphs with up to billions of nodes and edges, and we demonstrate that HoD significantly outperforms alternative solutions in terms of query efficiency.

• The paper introduces the Highways-on-Disk (HoD) technique as a disk-based index that efficiently processes single-source shortest path and distance queries on large, directed and weighted graphs.
• It employs an iterative node removal strategy with shortcut creation to preserve distances and optimize both memory usage and I/O operations.
• Performance evaluations demonstrate that HoD significantly reduces pre-computation time and query latency compared to existing methods, showing promise for large-scale real-world applications.

Efficient Single-Source Shortest Path and Distance Queries on Large Graphs

Introduction

The paper focuses on addressing the inefficiencies of current solutions for single-source shortest path (SSSP) and single-source distance (SSD) queries on large graphs, which frequently do not fit in main memory. The authors introduce "Highways-on-Disk" (HoD), a disk-based index structure that efficiently handles these queries on directed and weighted graphs, a departure from prior techniques focused solely on undirected and/or unweighted graphs.

Highways-on-Disk (HoD) Technique

HoD augments the original graph with auxiliary edges, known as shortcuts, to make use of these during query processing, reducing both I/O and computation costs. These shortcuts represent direct paths between nodes that bypass intermediary nodes while retaining the shortest path distance, thus streamlining the query process.

Index Construction

1. Node Selection and Shortcut Creation: Nodes are iteratively removed from the graph. Accompanying shortcuts are created to preserve direct distances between the neighbors of removed nodes. This process ensures that even after node removal, the essential topological structure allowing efficient distance computation remains intact.
2. Core Graph Determination: The reduction process continues until the remaining graph, referred to as the core graph, fits in main memory. This core graph is retained on disk, together with additional forward and backward graphs constructed from the remaining graph structure.
3. Memory and I/O Optimization: The entire indexing scheme involves sorting edges and arranging graph structures to facilitate linear scans during query processing, minimizing costly random disk accesses.

Query Processing

Queries are processed in three stages:

1. Forward Search: Initiate from the query's source node and traverse the augmented graph along higher-ranking paths, constrained by the order in which nodes are processed (increasing rank).
2. Core Search: Engage with the in-memory core graph, continuing to use a priority-based search to further expand paths discovered during the forward search.
3. Backward Search: Sequentially review the backward graph in reverse node order to refine distance calculations for nodes not included in the core.

This method ensures every query efficiently utilizes main memory for parts of the graph, complemented by disk-based operations carefully designed to emulate linear scans.

Performance Evaluation

HoD significantly reduces pre-computation time and space requirements when compared with existing solutions like VC-Index while achieving superior query response times. On undirected graphs, HoD is shown to outperform alternatives by margins that often reach an order of magnitude when handling massive datasets. Its ability to effectively handle directed graphs similarly marks a substantial improvement over existing methods constrained to undirected data.

Implications and Future Directions

HoD's focus on scalability and efficient disk usage presents it as a critical tool for real-world applications like social networks and map services utilizing massive graph data. Future work might explore extending this framework to accommodate dynamic graphs that evolve over time or integrating point-to-point shortest path queries within this disk-based framework, broadening its applicability in practical and varied scenarios.

In conclusion, HoD provides a robust foundation for SSD and SSSP queries on large-scale graph datasets, with significant potential to drive future advancements in large graph processing frameworks.