Add goes2go downloader functionality

spestana · Sep 25, 2024 · 3f2fc33 · 3f2fc33
1 parent 2869010
commit 3f2fc33
Show file tree

Hide file tree

Showing 6 changed files with 390 additions and 2,111 deletions.
diff --git a/docs/examples/download.ipynb b/docs/examples/download.ipynb
@@ -0,0 +1,217 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "3aea64b8-4c4c-478f-85a3-a5c36321c4c9",
+   "metadata": {},
+   "source": [
+    "# Download GOES\n",
+    "\n",
+    "This notebook demonstrates downloading a short time series of GOES-R ABI imagery."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "5c0d77eb-457b-4b4d-a669-73778a54637e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import goes_ortho as go\n",
+    "import xarray as xr"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0e01f57b-bbbe-4ebf-9c25-fff635326c2d",
+   "metadata": {},
+   "source": [
+    "First, specify the time range, location bounds, satellite, product (and if applicable, band and variable) that we'd like to access.\n",
+    "\n",
+    "We will also need to provide an API key for [OpenTopography.org](https://portal.opentopography.org/requestService?service=api) which you can create with a free account. This allows goes_ortho to access digital elevation models to perform the orthorectification step.\n",
+    "\n",
+    "The workflow below was developed to read a json file containing information about what we'd like to download. This was done to 1) allow these functions to run through github actions (still an experimental feature) and 2) keep a record of datasets we've downloaded. This is something that may change in the near future since it adds an unnecessary step for most use cases."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "6fc6c19c-f21f-4319-a45e-47434b933600",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "lon = -119.31212\n",
+    "lat = 37.88175\n",
+    "z = 2811  # elevation in meters"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "28208c5f-e93c-4ce9-b4f7-c4c7ea14e12e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "startDatetime = \"2024-04-19T00:00:00Z\"\n",
+    "endDatetime = \"2024-04-19T00:59:00Z\"\n",
+    "[min_lon, min_lat, max_lon, max_lat] = [lon-1, lat-1, lon+1, lat+1] # set bounds to just a little area around the location we want\n",
+    "satellite = \"goes18\"\n",
+    "product = \"ABI-L1b-RadC\"\n",
+    "band = 2\n",
+    "variable = \"Rad\"\n",
+    "OPENTOPO_API_KEY = \"585b1d1639bc5ef8a4a5bdea7e45a8d1\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8a4c97cb-b630-4771-8e80-4c7f70c0f1bf",
+   "metadata": {},
+   "source": [
+    "Using the data above, we make the required json file."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "cac7ae71-01db-4fa3-9e9f-856b82ff738b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Make request file from user input\n",
+    "go.get_data.make_request_json(\n",
+    "    f\"{satellite}-example-ACMC\",\n",
+    "    startDatetime,\n",
+    "    endDatetime,\n",
+    "    [min_lon, min_lat, max_lon, max_lat],\n",
+    "    satellite,\n",
+    "    product,\n",
+    "    band,\n",
+    "    variable,\n",
+    "    OPENTOPO_API_KEY,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "9fbfaf9c-938a-475a-a1f7-f2a36b062505",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Estimated 1 batches to download\n",
+      "Batch number 1\n",
+      "Download batch of imagery from 2024-04-19 00:00:00+00:00 to 2024-04-19 03:00:00+00:00\n",
+      "📦 Finished downloading [576] files to [/home/spestana/data/noaa-goes18/ABI-L1b-RadC].\n",
+      "Cropping image batch to [-120.31212, 36.88175, -118.31212, 38.88175]\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "100%|██████████| 576/576 [01:44<00:00,  5.52it/s]"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Done\n",
+      "CPU times: user 1min 58s, sys: 22 s, total: 2min 20s\n",
+      "Wall time: 2min 50s\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%time\n",
+    "filepaths = go.get_data.download_abi_goes2go(f\"{satellite}-example-ACMC.json\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "738b548c-8807-4fb8-82c2-eb380a5284c8",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Files will be downloaded and then cropped to these bounds:\n",
+      "\t(-120.31212,38.88175).\t.(-118.31212,38.88175)\n",
+      "\n",
+      "\n",
+      "\n",
+      "\t(-120.31212,36.88175).\t.(-118.31212,36.88175)\n",
+      "\n",
+      "For each S3 bucket, download the corresponding observations\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02\n",
+      "Downloading... the product ABI-L1b-RadC \n",
+      "Downloading dataset to... 00 UTC\n",
+      "OR_ABI-L1b-RadC-M6C02_G18_s20241100056172_e20241100058545_c20241100058573.nc: [##########] 100%  67.930455 MB/67.930455 MB\n",
+      "Subsetting files in...goes18/2024/4/19/ABI-L1b-RadC/00/C02\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100036172_e20241100038545_c20241100038570.nc\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100006172_e20241100008545_c20241100008570.nc\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100046172_e20241100048545_c20241100048571.nc\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100021172_e20241100023545_c20241100023570.nc\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100011172_e20241100013545_c20241100013573.nc\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100016172_e20241100018545_c20241100018577.nc\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100026172_e20241100028545_c20241100028580.nc\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100031172_e20241100033545_c20241100033572.nc\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100056172_e20241100058545_c20241100058573.nc\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100001172_e20241100003545_c20241100003572.nc\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100041172_e20241100043545_c20241100043570.nc\n",
+      "goes18/2024/4/19/ABI-L1b-RadC/00/C02/OR_ABI-L1b-RadC-M6C02_G18_s20241100051172_e20241100053545_c20241100053568.nc\n",
+      "Done\n",
+      "CPU times: user 15.7 s, sys: 4.8 s, total: 20.5 s\n",
+      "Wall time: 51 s\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%time\n",
+    "filepaths = go.get_data.download_abi_goespy(f\"{satellite}-example-ACMC.json\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "bd893536-d241-4966-a49c-dd93729d6c43",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "goes-test-env",
+   "language": "python",
+   "name": "goes-test-env"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.5"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/docs/examples/example.json b/docs/examples/example.json