-
Notifications
You must be signed in to change notification settings - Fork 13
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
1a013db
commit 6112a35
Showing
3 changed files
with
35 additions
and
55 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,29 +1,45 @@ | ||
--- | ||
title: 'Group Project: Data Processing' | ||
title: 'Group Project: Visualization' | ||
--- | ||
|
||
|
||
## Setup | ||
|
||
In your fork of the [`scalable-computing-examples`](https://github.com/NCEAS/scalable-computing-examples) repository, open the Jupyter notebook in the `group-project` directory called `session-11.ipynb`. This workbook will serve as a skeleton for you to work in. It will load in all the libraries you need, including a few helper functions we wrote for the course, show an example for how to use the method on one file, and then lays out blocks for you and your group to fill in with code that will run that method in parallel. | ||
In your fork of the [`scalable-computing-examples`](https://github.com/NCEAS/scalable-computing-examples) repository, open the Jupyter notebook in the `group-project` directory called `session-15.ipynb`. This workbook will serve as a skeleton for you to work in. It will load in all the libraries you need, including a few helper functions we wrote for the course, show an example for how to use the method on one file, and then lays out blocks for you and your group to fill in with code that will run that method in parallel. | ||
|
||
In your small groups, work together to write the solution, but everyone should aim to have a working solution on their own fork of the repository. In other words, everyone should type out the solution themselves as part of the group effort. Writing the code out yourself (even if others are contributing to the content) is a great way to get "mileage" as you develop these skills. | ||
|
||
Only one person in the group should run the parallel code. | ||
|
||
## Rasterizing GeoPackages | ||
## Resampling rasters | ||
|
||
In this portion of the group project, we will further process the raster files by resampling them to lower zoom levels. This means we will take the average of clusters of pixels, and that value will be assigned to the single pixel that encompasses the entire area that is represented by the original cluster of pixels. Here is an example of creating a zoom level 10 pixel from 4 zoom level 11 pixels: | ||
|
||
![Drawing of downsampling a raster, created by Robyn Thiessen-Bock.](../images/resampling.png) | ||
|
||
As we aggregate cells to produce rasters at lower zoom levels, the _cell size increases_, but the _extent of the raster remains the same_. This is called "downsampling". | ||
|
||
After we do this for each zoom level, you can imagine they are organzied as a pyramid, with the lowest zoom levels (and lowest resolution rasters) at the top, like so: | ||
|
||
![A pyramid of rasters at different resolutions (zoom levels). Souce: [maptiler](https://www.maptiler.com/google-maps-coordinates-tile-bounds-projection/#3/15.00/50.00)](../images/raster_pyramid.png) | ||
|
||
The highest zoom level has the most rasters. As we create the lower zoom levels, we aggregate the rasters _only_ from the zoom level directly above. As the zoom level decreases, our computation is faster. | ||
|
||
Check out the [Permafrost Discovery Gateway Imagery Viewer]() and zoom in and out. You can see for yourself how the resolution increases and the extent that you are trying to view decreases. | ||
|
||
In the last lession, we staged the input files into GeoPackages. Now we will import those `.gpkg` files and write each as a raster (`.tif`). The resulting rasters will have 2 bands, one for each statistic we calculate based on the vector geometries in each pixel. | ||
![Ice wedge polygons at high resolution, visible when zoomed-in to the Permafrost Discovery Gateway Imagery Viewer.](../images/zoom_in_IWP.png) | ||
|
||
We need to create the highest zoom level rasters before we create the lower zoom levels in the next lesson. This brings us one step closer to visualizing the ice wedge polygons on a basemap, with the ability to zoom in and out! | ||
The higher resolution tiles are the "child" tiles, and the lower resolution tiles are the "parent" tiles. | ||
|
||
### Packages, Libraries, and Modules | ||
|
||
- [os 3.11.2](https://docs.python.org/3/library/os.html) | ||
- [parsl 2023.3.20](https://parsl.readthedocs.io/en/stable/) | ||
- [pdgstaging](https://github.com/PermafrostDiscoveryGateway/viz-staging) | ||
- package developed by Permafrost Discovery Gateway software developer and designer Robyn Thiessen-Bock | ||
- developed by Permafrost Discovery Gateway software developer and designer Robyn Thiessen-Bock | ||
- [pdgraster](https://github.com/PermafrostDiscoveryGateway/viz-raster/tree/main) | ||
- package developed by Permafrost Discovery Gateway software developer and designer Robyn Thiessen-Bock | ||
- developed by Permafrost Discovery Gateway software developer and designer Robyn Thiessen-Bock | ||
- [geopandas0.11](https://geopandas.org/en/stable/docs.html) | ||
- [random](https://docs.python.org/3/library/random.html) | ||
- [matplotlib 3.5](https://matplotlib.org/3.5.3/api/_as_gen/matplotlib.pyplot.html) | ||
- [matplotlib 3.5](https://matplotlib.org/3.5.3/api/_as_gen/matplotlib.pyplot.html) | ||
- [ipyleaflet 0.17](https://ipyleaflet.readthedocs.io/en/latest/) |
This file was deleted.
Oops, something went wrong.