diff --git a/notebooks/cugraph_benchmarks/synth_release_single_node_multi_gpu.ipynb b/notebooks/cugraph_benchmarks/synth_release_single_node_multi_gpu.ipynb new file mode 100644 index 00000000000..c44f475c441 --- /dev/null +++ b/notebooks/cugraph_benchmarks/synth_release_single_node_multi_gpu.ipynb @@ -0,0 +1,950 @@ +{ + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "# Skip notebook test\n", + "-----\n", + "\n", + "#### NOTE: This notebook will take hours to run.\n", + "-----\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "# Comparing NetworkX vs cuGraph using synthetic data on various algorithms on single node multi GPU (SNMG) cluster\n", + "\n", + "\n", + "This notebook compares the execution times of many of the cuGraph and NetworkX algorithms when run against identical synthetic data at multiple scales.\n", + "\n", + "This notebook uses the RMAT data generator which allows the creation of graphs at various scales. The notebook, by default, runs on a set of selected sizes but users are free to change or add to that list." + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notebook Credits\n", + "\n", + " \n", + "| Author | Date | Update | cuGraph Version | Test Hardware |\n", + "| --------------|------------|---------------------|-----------------|------------------------|\n", + "| Don Acosta | 1/12/2023 | Created | 23.02 nightly | RTX A6000, CUDA 11.7 |\n", + "| Brad Rees | 1/27/2023 | Modified | 23.02 nightly | RTX A6000, CUDA 11.7 |\n", + "| Naim, Md | 2/08/2024 | Modified for SNMG | 24.04 nightly | RTX A6000, CUDA 12.0 |\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "### Timing " + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When looking at the overall workflow, NetworkX and cuGraph do things differently. For example, NetworkX spends a lot of time creating the graph data structure. cuGraph on the other hand does a lazy creation of the data structure when an algorithm is called. To further complicate the comparison problem, NetworkX does not always return the answer. In some cases, it returns a generator that is then called to produce the data. \n", + "\n", + "This benchmark produces two performance metrics:\n", + " - (1)\tJust the algorithm run time \n", + " - (2)\tThe algorithm plus graph creation time\n", + "\n", + "Since GPU memory is a precious resource, having a lot of temporary data laying around is avoided. So once a graph is created, the raw data is dropped. \n", + " \n", + "__What is not timed__: Generating the data with R-MAT

\n", + "__What is timed__: (1) creating a Graph, (2) running the algorithm (3) run any generators\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Algorithms" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "| Algorithm | Type | Undirected Graph | Directed Graph | Notes\n", + "| ------------------------|---------------|------ | ------- |-------------\n", + "| Katz | Centrality | X | | \n", + "| Betweenness Centrality | Centrality | X | | Estimated, k = 100\n", + "| Louvain | Community | X | | Uses python-louvain for comparison\n", + "| Triangle Counting | Community | X | |\n", + "| Core Number | Core | X | |\n", + "| PageRank | Link Analysis | | X |\n", + "| Jaccard | Similarity | X | |\n", + "| BFS | Traversal | X | | No depth limit\n", + "| SSSP | Traversal | X | | \n", + "\n", + "\n", + "### Test Data\n", + "Data is generated using a Recursive MATrix (R-MAT) graph generation algorithm. \n", + "The generator specifics are documented [here](https://docs.rapids.ai/api/cugraph/stable/api_docs/generator.html)\n", + "\n", + "\n", + "\n", + "### Notes\n", + "* Running Betweenness Centrality on the full graph is prohibitive using NetworkX. Anything over k=100 can explode runtime to days\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Import Modules" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# system and other\n", + "import gc\n", + "import os\n", + "from time import perf_counter\n", + "import pandas as pd\n", + "from collections import defaultdict\n", + "\n", + "# rapids\n", + "import cugraph\n", + "\n", + "# liblibraries to setup dask cluster and client\n", + "from dask.distributed import Client\n", + "from dask_cuda import LocalCUDACluster\n", + "from cugraph.dask.comms import comms as Comms\n", + "\n", + "# NetworkX libraries\n", + "import networkx as nx\n", + "\n", + "# RMAT data generator\n", + "from cugraph.generators import rmat\n", + "from cugraph.structure import NumberMap" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "try: \n", + " import community\n", + "except ModuleNotFoundError:\n", + " os.system('pip install python-louvain')\n", + " import community" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Determine the scale of the test data\n", + "RMAT generates graph where the number of vertices is a power of 2 and the number of edges is based on an edge factor times the number vertices.\n", + "\n", + "Since RMAT tends to generate about 50% isolated vertices, those vertices are dropped from the graph data. Hence the number of vertices is closer to (2 ** scale) / 2\n", + "\n", + "\n", + "| Scale | Vertices (est) | Edges |\n", + "| ------|----------------|--------|\n", + "| 10 | 512 | 16,384 | \n", + "| 11 | 1,024 | 32,768| \n", + "| 12 | 2,048 | 65,536| \n", + "| 13 | 4,096 | 131,072| \n", + "| 14 | 8,192 | 262,144| \n", + "| 15 | 16,384 | 524,288 | \n", + "| 16 | 32,768 | 1,048,576 | \n", + "| 17 | 65,536 | 2,097,152 | \n", + "| 18 | 131,072 | 4,194,304 | \n", + "| 19 | 262,144 | 8,388,608 | \n", + "| 20 | 524,288 | 16,777,216 | \n", + "| 21 | 1,048,576 | 33,554,432 | \n", + "| 22 | 2,097,152 | 67,108,864 | \n", + "| 23 | 4,194,304 | 134,217,728 | \n", + "| 24 | 8,388,608 | 268,435,456 | \n", + "| 25 | 16,777,216 | 536,870,912 | \n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Test Data Sizes\n", + "# Here you can create an array of test data sizes. Then set the \"data\" variable to the array you want\n", + "# the dictionary format is 'name' : scale\n", + "\n", + "\n", + "# These scales are used by R-MAT to determine the number of vertices/edges in the synthetic data graph.\n", + "data_full = {\n", + " 'data_scale_10' : 10,\n", + " 'data_scale_12' : 12,\n", + " 'data_scale_14' : 14,\n", + " 'data_scale_16' : 16,\n", + " 'data_scale_18' : 18,\n", + " 'data_scale_20' : 20,\n", + "}\n", + "\n", + "# for quick testing\n", + "data_quick = {\n", + " 'data_scale_9' : 9,\n", + " 'data_scale_10' : 10,\n", + " 'data_scale_11' : 11,\n", + "}\n", + "\n", + "\n", + "# Which dataset is to be used\n", + "data = data_quick\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Generate data\n", + "The data is generated once for each size." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Data generator \n", + "# The result is an edgelist of the size determined by the scale and edge factor\n", + "def generate_data(scale, edgefactor=16, mg=False):\n", + " _gdf = rmat(\n", + " scale,\n", + " (2 ** scale) * edgefactor,\n", + " 0.57,\n", + " 0.19,\n", + " 0.19,\n", + " 42,\n", + " clip_and_flip=False,\n", + " scramble_vertex_ids=True,\n", + " create_using=None, # return edgelist instead of Graph instance\n", + " mg=mg # determines whether generated data will be used on one or multiple GPUs\n", + " )\n", + "\n", + " clean_coo = NumberMap.renumber(_gdf, src_col_names=\"src\", dst_col_names=\"dst\")[0]\n", + " if mg:\n", + " clean_coo.rename(columns={\"renumbered_src\": \"src\", \"renumbered_dst\": \"dst\"})\n", + " else:\n", + " clean_coo.rename(columns={\"renumbered_src\": \"src\", \"renumbered_dst\": \"dst\"}, inplace=True)\n", + "\n", + " print(f'Generated a dataframe of type {type(clean_coo)}, with {len(clean_coo)} edges')\n", + " \n", + " return clean_coo" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create Graph functions\n", + "There are two types of graphs created:\n", + "* Directed Graphs - calls to create_nx_digraph, create_cu_directed_graph.\n", + "* Undirected Graphs - calls to create_xx_ugraph <- fully symmeterized" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# NetworkX\n", + "def create_nx_graph(_df , directed=False):\n", + " t1 = perf_counter()\n", + " if directed:\n", + " g_type = nx.DiGraph\n", + " else:\n", + " g_type = nx.Graph\n", + " \n", + " _gnx = nx.from_pandas_edgelist(_df,\n", + " source='src',\n", + " target='dst',\n", + " edge_attr=None,\n", + " create_using=g_type)\n", + " t2 = perf_counter() - t1\n", + "\n", + " return _gnx, t2\n", + "\n", + "\n", + "\n", + "# cuGraph\n", + "def create_cu_graph(_df, transpose=False, directed=False, mg=False):\n", + " t1 = perf_counter()\n", + " _g = cugraph.Graph(directed=directed)\n", + "\n", + " if mg:\n", + " _g.from_dask_cudf_edgelist(_df, source=\"src\", destination=\"dst\", edge_attr=None)\n", + " else:\n", + " _g.from_cudf_edgelist(_df,\n", + " source='src',\n", + " destination='dst',\n", + " edge_attr=None,\n", + " renumber=False,\n", + " store_transposed=transpose)\n", + " t2 = perf_counter() - t1\n", + "\n", + " return _g, t2" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Algorithm Execution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Katz" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def nx_katz(_G, alpha):\n", + " t1 = perf_counter()\n", + " _ = nx.katz_centrality(_G, alpha)\n", + " t2 = perf_counter() - t1\n", + " return t2\n", + "\n", + "def cu_katz(_G, alpha, mg=False):\n", + " t1 = perf_counter()\n", + " if mg:\n", + " _ = cugraph.dask.katz_centrality(_G, alpha)\n", + " else:\n", + "\n", + " _ = cugraph.katz_centrality(_G, alpha)\n", + " t2 = perf_counter() - t1\n", + " return t2\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Betweenness Centrality" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def nx_bc(_G, _k):\n", + " t1 = perf_counter()\n", + " _ = nx.betweenness_centrality(_G, k=_k)\n", + " t2 = perf_counter() - t1\n", + " return t2\n", + "\n", + "def cu_bc(_G, _k, mg=False):\n", + " t1 = perf_counter()\n", + " if mg:\n", + " _ = cugraph.dask.betweenness_centrality(_G, k=_k)\n", + " else: \n", + " _ = cugraph.betweenness_centrality(_G, k=_k)\n", + " t2 = perf_counter() - t1\n", + " return t2\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Louvain" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def nx_louvain(_G):\n", + " t1 = perf_counter()\n", + " parts = community.best_partition(_G)\n", + " \n", + " # Calculating modularity scores for comparison\n", + " _ = community.modularity(parts, _G)\n", + " \n", + " t2 = perf_counter() - t1\n", + " return t2\n", + "\n", + "def cu_louvain(_G, mg=False):\n", + " t1 = perf_counter()\n", + " if mg:\n", + " _, modularity = cugraph.dask.louvain(_G)\n", + " print (f'modularity: {modularity}')\n", + " else:\n", + " _,_ = cugraph.louvain(_G)\n", + " t2 = perf_counter() - t1\n", + " return t2\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Triangle Counting" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def nx_tc(_G):\n", + " t1 = perf_counter()\n", + " nx_count = nx.triangles(_G)\n", + "\n", + " # To get the number of triangles, we would need to loop through the array and add up each count\n", + " count = 0\n", + " for key, value in nx_count.items():\n", + " count = count + value\n", + " \n", + " t2 = perf_counter() - t1\n", + " return t2\n", + "\n", + "def cu_tc(_G, mg=False):\n", + " t1 = perf_counter()\n", + " if mg:\n", + " _ = cugraph.dask.triangle_count(_G)\n", + " else:\n", + " _ = cugraph.triangle_count(_G)\n", + " t2 = perf_counter() - t1\n", + " return t2\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Core Number" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def nx_core_num(_G):\n", + " t1 = perf_counter()\n", + " _G.remove_edges_from(nx.selfloop_edges(_G))\n", + " nx_count = nx.core_number(_G)\n", + " \n", + " count = 0\n", + " for key, value in nx_count.items():\n", + " count = count + value\n", + " \n", + " t2 = perf_counter() - t1\n", + " return t2\n", + "\n", + "def cu_core_num(_G, mg=False):\n", + " t1 = perf_counter()\n", + " if mg:\n", + " _ = cugraph.dask.core_number(_G)\n", + " else:\n", + " _ = cugraph.core_number(_G)\n", + " t2 = perf_counter() - t1\n", + " return t2\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### PageRank" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def nx_pagerank(_G):\n", + " t1 = perf_counter()\n", + " _ = nx.pagerank(_G)\n", + " t2 = perf_counter() - t1\n", + " return t2 \n", + "\n", + "def cu_pagerank(_G, mg=False):\n", + " t1 = perf_counter()\n", + " if mg:\n", + " _ = cugraph.dask.pagerank(_G)\n", + " else:\n", + " _ = cugraph.pagerank(_G)\n", + " t2 = perf_counter() - t1\n", + " return t2\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Jaccard" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def nx_jaccard(_G):\n", + " t1 = perf_counter()\n", + " nj = nx.jaccard_coefficient(_G)\n", + " t2 = perf_counter() - t1\n", + " return t2\n", + "\n", + "def cu_jaccard(_G, mg=False):\n", + " t1 = perf_counter()\n", + " t1 = perf_counter()\n", + " if mg:\n", + " _ = cugraph.dask.jaccard(_G)\n", + " else:\n", + " _ = cugraph.jaccard_coefficient(_G)\n", + " t2 = perf_counter() - t1\n", + " return t2\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### BFS" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def nx_bfs(_G, seed):\n", + " t1 = perf_counter()\n", + " nb = nx.bfs_edges(_G, seed)\n", + " nb_list = list(nb) # gen -> list\n", + " t2 = perf_counter() - t1\n", + " return t2\n", + "\n", + "def cu_bfs(_G, seed=0, mg=False):\n", + " t1 = perf_counter()\n", + " if mg:\n", + " _ = cugraph.dask.bfs(_G, seed)\n", + " else:\n", + " _ = cugraph.bfs(_G, seed)\n", + " t2 = perf_counter() - t1\n", + " return t2\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### SSSP" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def nx_sssp(_G, seed):\n", + " t1 = perf_counter()\n", + " _ = nx.shortest_path(_G, seed)\n", + " t2 = perf_counter() - t1\n", + " return t2\n", + "\n", + "def cu_sssp(_G, seed = 0, mg=False):\n", + " \n", + " t1 = perf_counter()\n", + " # SSSP requires weighted graph\n", + " if mg:\n", + " _ = cugraph.dask.bfs(_G, seed)\n", + " else:\n", + " _ = cugraph.bfs(_G, seed)\n", + " t2 = perf_counter() - t1\n", + " return t2\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## MG Benchmark" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Initialize multi-GPU environment\n", + "Before we get started, we need to set up a dask (local) cluster of workers to execute our work, and a client to coordinate and schedule work for that cluster.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Setup a local dask cluster of workers, and a client\n", + "cluster = LocalCUDACluster()\n", + "client = Client(cluster)\n", + "Comms.initialize(p2p=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Placeholders to collect execution run statistics " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "\n", + "nx_algo_run_times = defaultdict(defaultdict)\n", + "cugraph_algo_run_times = defaultdict(defaultdict)\n", + "perf_algos = defaultdict(defaultdict)\n", + "perf = defaultdict(defaultdict)\n", + "cugraph_graph_creation_times = defaultdict()\n", + "nx_graph_creation_times = defaultdict()\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Run NX and cuGraph algorithms for all datasets" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "for dataset, scale in data.items():\n", + " \n", + " # generate data\n", + " print(\"------------------------------\")\n", + " print(f'Creating Graph of Scale = {scale}')\n", + " \n", + " gdf = generate_data(scale, edgefactor=16, mg=True)\n", + " gdf = gdf.repartition(gdf.npartitions * 3)\n", + "\n", + " # Copy data to host to create NX graph\n", + " pdf = pd.DataFrame(columns=['src', 'dst'])\n", + " for part_idx in range(gdf.npartitions):\n", + " computed_df = gdf.partitions[part_idx].compute().to_pandas()\n", + " pdf = pd.concat([pdf, computed_df], ignore_index=True, sort=False)\n", + "\n", + " print(f\"\\tdata in gdf {len(gdf)} and data in pandas {len(pdf)}\")\n", + " \n", + " # create cuGraph and NX graphs\n", + " g_cu, tcu = create_cu_graph(gdf, mg=True)\n", + " g_nx, tnx = create_nx_graph(pdf)\n", + " cugraph_graph_creation_times[dataset] = tcu\n", + " nx_graph_creation_times[dataset] = tnx\n", + " del gdf, pdf\n", + "\n", + " # prep\n", + " deg = g_cu.degree()\n", + " deg_max = deg['degree'].max().compute()\n", + " alpha = 1 / deg_max\n", + " num_nodes = g_cu.number_of_vertices()\n", + " del deg\n", + " gc.collect()\n", + "\n", + " #-- Katz \n", + " algorithm = \"Katz\"\n", + " print(f\"\\t{algorithm} \", end = '')\n", + " print(\"n.\", end='')\n", + " tx = nx_katz(g_nx, alpha)\n", + " print(\"c.\", end='')\n", + " tc = cu_katz(g_cu, alpha, mg=True)\n", + " print(\"\")\n", + " \n", + " nx_algo_run_times[dataset][algorithm] = tx\n", + " cugraph_algo_run_times[dataset][algorithm] = tc\n", + " perf_algos[dataset][algorithm] = tx/tc \n", + " perf[dataset][algorithm] = (tx + tnx) / (tc + tcu)\n", + "\n", + " #-- BC\n", + " algorithm = \"BC\"\n", + " print(f\"\\t{algorithm} \", end = '')\n", + " k = 100\n", + " if k > num_nodes:\n", + " k = int(num_nodes)\n", + " print(\"n.\", end='')\n", + " tx = nx_bc(g_nx, k)\n", + " print(\"c.\", end='')\n", + " tc = cu_bc(g_cu, k, mg=True)\n", + " print(\" \")\n", + " nx_algo_run_times[dataset][algorithm] = tx\n", + " cugraph_algo_run_times[dataset][algorithm] = tc\n", + " perf_algos[dataset][algorithm] = tx/tc \n", + " perf[dataset][algorithm] = (tx + tnx) / (tc + tcu)\n", + "\n", + " #-- Louvain\n", + " algorithm = \"Louvain\"\n", + " print(f\"\\t{algorithm} \", end = '')\n", + " print(\"n.\", end='')\n", + " tx = nx_louvain(g_nx)\n", + " print(\"c.\", end='')\n", + " tc = cu_louvain(g_cu, mg=True)\n", + " print(\" \")\n", + "\n", + " nx_algo_run_times[dataset][algorithm] = tx\n", + " cugraph_algo_run_times[dataset][algorithm] = tc\n", + " perf_algos[dataset][algorithm] = tx/tc \n", + " perf[dataset][algorithm] = (tx + tnx) / (tc + tcu)\n", + "\n", + " #-- TC\n", + " algorithm = \"TC\"\n", + " print(f\"\\t{algorithm} \", end = '')\n", + " print(\"n.\", end='')\n", + " tx = nx_tc(g_nx)\n", + " print(\"c.\", end='')\n", + " tc = cu_tc(g_cu, mg=True)\n", + " print(\" \")\n", + " \n", + " nx_algo_run_times[dataset][algorithm] = tx\n", + " cugraph_algo_run_times[dataset][algorithm] = tc\n", + " perf_algos[dataset][algorithm] = tx/tc \n", + " perf[dataset][algorithm] = (tx + tnx) / (tc + tcu)\n", + "\n", + " #-- Core Number\n", + " algorithm = \"Core Number\"\n", + " print(f\"\\t{algorithm} \", end = '')\n", + " print(\"n.\", end='')\n", + " tx = nx_core_num(g_nx)\n", + " print(\"c.\", end='')\n", + " tc = cu_core_num(g_cu, mg=True)\n", + " print(\" \")\n", + "\n", + " nx_algo_run_times[dataset][algorithm] = tx\n", + " cugraph_algo_run_times[dataset][algorithm] = tc\n", + " perf_algos[dataset][algorithm] = tx/tc \n", + " perf[dataset][algorithm] = (tx + tnx) / (tc + tcu)\n", + "\n", + " #-- PageRank\n", + " algorithm = \"PageRank\"\n", + " print(f\"\\t{algorithm} \", end = '')\n", + " print(\"n.\", end='')\n", + " tx = nx_pagerank(g_nx)\n", + " print(\"c.\", end='')\n", + " tc = cu_pagerank(g_cu, mg=True)\n", + " print(\" \")\n", + "\n", + " nx_algo_run_times[dataset][algorithm] = tx\n", + " cugraph_algo_run_times[dataset][algorithm] = tc\n", + " perf_algos[dataset][algorithm] = tx/tc \n", + " perf[dataset][algorithm] = (tx + tnx) / (tc + tcu)\n", + "\n", + " #-- Jaccard\n", + " algorithm = \"Jaccard\"\n", + " print(f\"\\t{algorithm} \", end = '')\n", + "\n", + " print(\"n.\", end='')\n", + " tx = nx_jaccard(g_nx)\n", + " print(\"c.\", end='')\n", + " tc = cu_jaccard(g_cu, mg=True)\n", + " print(\" \")\n", + "\n", + " nx_algo_run_times[dataset][algorithm] = tx\n", + " cugraph_algo_run_times[dataset][algorithm] = tc\n", + " perf_algos[dataset][algorithm] = tx/tc \n", + " perf[dataset][algorithm] = (tx + tnx) / (tc + tcu)\n", + "\n", + " # Seed for BFS and SSSP\n", + " nx_seed = list(g_nx.nodes)[0]\n", + " cu_seed = g_cu.nodes().compute().to_pandas().iloc[0]\n", + "\n", + " #-- BFS\n", + " algorithm = \"BFS\"\n", + " print(f\"\\t{algorithm} \", end = '')\n", + " print(\"n.\", end='')\n", + " tx = nx_bfs(g_nx, nx_seed)\n", + " print(\"c.\", end='')\n", + " tc = cu_bfs(g_cu, seed = cu_seed, mg=True)\n", + " print(\" \")\n", + "\n", + " nx_algo_run_times[dataset][algorithm] = tx\n", + " cugraph_algo_run_times[dataset][algorithm] = tc\n", + " perf_algos[dataset][algorithm] = tx/tc \n", + " perf[dataset][algorithm] = (tx + tnx) / (tc + tcu)\n", + "\n", + " #-- SSSP\n", + " algorithm = \"SSSP\"\n", + " print(f\"\\t{algorithm} \", end = '')\n", + " print(\"n.\", end='')\n", + " tx = nx_sssp(g_nx, nx_seed)\n", + "\n", + " print(\"c.\", end='')\n", + " tc = cu_sssp(g_cu, seed = cu_seed, mg=True)\n", + " print(\" \")\n", + "\n", + " nx_algo_run_times[dataset][algorithm] = tx\n", + " cugraph_algo_run_times[dataset][algorithm] = tc\n", + " perf_algos[dataset][algorithm] = tx/tc \n", + " perf[dataset][algorithm] = (tx + tnx) / (tc + tcu)\n", + "\n", + " del g_cu, g_nx\n", + " gc.collect()\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### cuGraph speedup of different algorithms w.r.t. NX" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(\"\\n\\t------Speedup (cuGraph w.r.t. NX)------\\n\")\n", + "print(pd.DataFrame(perf))\n", + "print(\"\\n\\t------Speedup (cuGraph w.r.t. NX, excluding graph creation time)------\\n\")\n", + "print(pd.DataFrame(perf_algos))\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Nx and cuGraph execution times for different algorithms" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "nx_and_cugraph_run_times = pd.DataFrame()\n", + "for dataset in cugraph_algo_run_times.keys():\n", + " temp_df = pd.DataFrame({'NX': nx_algo_run_times[dataset], 'cuGraph': cugraph_algo_run_times[dataset]})\n", + " columns = [(dataset, 'cuGraph'), (dataset, 'NX')]\n", + " temp_df.columns = pd.MultiIndex.from_tuples(columns)\n", + " nx_and_cugraph_run_times = pd.concat([temp_df, nx_and_cugraph_run_times], axis=1)\n", + "\n", + "print(\"\\n\\t------Nx and cuGraph execution times for different algorithms-----\\n\")\n", + "print(nx_and_cugraph_run_times)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Clean up multi-GPU environment" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "Comms.destroy()\n", + "client.close()\n", + "cluster.close()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "___\n", + "Copyright (c) 2020-2023, NVIDIA CORPORATION.\n", + "\n", + "Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0\n", + "\n", + "Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n", + "___" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "cudfdev", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.1.0" + }, + "vscode": { + "interpreter": { + "hash": "587ff963ecd34554a9da41c94362e2baa062d9a57502e220f049e10816826984" + } + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}