Skip to content

Commit

Permalink
combine group projects
Browse files Browse the repository at this point in the history
  • Loading branch information
jeanetteclark committed Mar 21, 2024
1 parent 1a013db commit 6112a35
Show file tree
Hide file tree
Showing 3 changed files with 35 additions and 55 deletions.
13 changes: 11 additions & 2 deletions sections/group-project-1.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,10 @@ In the final step of the workflow, we will take the raster files and resample th

![](../images/gp-step3.png)

## Staging and Tiling

Today we will undertake the first step of the workflow, staging and tiling the data.
Today we will undertake the first two steps of the workflow, staging and tiling the data, and then rasterizing it.

## Staging and Tiling

![](../images/gp-step1.png)

Expand All @@ -45,12 +46,20 @@ In your small groups, work together to write the solution, but everyone should a

Only one person in the group should run the parallel code.

## Rasterizing GeoPackages

In the last step, we staged the input files into GeoPackages. Now we will import those `.gpkg` files and write each as a raster (`.tif`). The resulting rasters will have 2 bands, one for each statistic we calculate based on the vector geometries in each pixel.

We need to create the highest zoom level rasters before we create the lower zoom levels in the next lesson. This brings us one step closer to visualizing the ice wedge polygons on a basemap, with the ability to zoom in and out!

### Packages, Libraries, and Modules

- [os 3.11.2](https://docs.python.org/3/library/os.html)
- [parsl 2023.3.20](https://parsl.readthedocs.io/en/stable/)
- [pdgstaging](https://github.com/PermafrostDiscoveryGateway/viz-staging)
- package developed by Permafrost Discovery Gateway software developer and designer Robyn Thiessen-Bock
- [pdgraster](https://github.com/PermafrostDiscoveryGateway/viz-raster/tree/main)
- package developed by Permafrost Discovery Gateway software developer and designer Robyn Thiessen-Bock
- [geopandas0.11](https://geopandas.org/en/stable/docs.html)
- [random](https://docs.python.org/3/library/random.html)
- [matplotlib 3.5](https://matplotlib.org/3.5.3/api/_as_gen/matplotlib.pyplot.html)
32 changes: 24 additions & 8 deletions sections/group-project-2.qmd
Original file line number Diff line number Diff line change
@@ -1,29 +1,45 @@
---
title: 'Group Project: Data Processing'
title: 'Group Project: Visualization'
---


## Setup

In your fork of the [`scalable-computing-examples`](https://github.com/NCEAS/scalable-computing-examples) repository, open the Jupyter notebook in the `group-project` directory called `session-11.ipynb`. This workbook will serve as a skeleton for you to work in. It will load in all the libraries you need, including a few helper functions we wrote for the course, show an example for how to use the method on one file, and then lays out blocks for you and your group to fill in with code that will run that method in parallel.
In your fork of the [`scalable-computing-examples`](https://github.com/NCEAS/scalable-computing-examples) repository, open the Jupyter notebook in the `group-project` directory called `session-15.ipynb`. This workbook will serve as a skeleton for you to work in. It will load in all the libraries you need, including a few helper functions we wrote for the course, show an example for how to use the method on one file, and then lays out blocks for you and your group to fill in with code that will run that method in parallel.

In your small groups, work together to write the solution, but everyone should aim to have a working solution on their own fork of the repository. In other words, everyone should type out the solution themselves as part of the group effort. Writing the code out yourself (even if others are contributing to the content) is a great way to get "mileage" as you develop these skills.

Only one person in the group should run the parallel code.

## Rasterizing GeoPackages
## Resampling rasters

In this portion of the group project, we will further process the raster files by resampling them to lower zoom levels. This means we will take the average of clusters of pixels, and that value will be assigned to the single pixel that encompasses the entire area that is represented by the original cluster of pixels. Here is an example of creating a zoom level 10 pixel from 4 zoom level 11 pixels:

![Drawing of downsampling a raster, created by Robyn Thiessen-Bock.](../images/resampling.png)

As we aggregate cells to produce rasters at lower zoom levels, the _cell size increases_, but the _extent of the raster remains the same_. This is called "downsampling".

After we do this for each zoom level, you can imagine they are organzied as a pyramid, with the lowest zoom levels (and lowest resolution rasters) at the top, like so:

![A pyramid of rasters at different resolutions (zoom levels). Souce: [maptiler](https://www.maptiler.com/google-maps-coordinates-tile-bounds-projection/#3/15.00/50.00)](../images/raster_pyramid.png)

The highest zoom level has the most rasters. As we create the lower zoom levels, we aggregate the rasters _only_ from the zoom level directly above. As the zoom level decreases, our computation is faster.

Check out the [Permafrost Discovery Gateway Imagery Viewer]() and zoom in and out. You can see for yourself how the resolution increases and the extent that you are trying to view decreases.

In the last lession, we staged the input files into GeoPackages. Now we will import those `.gpkg` files and write each as a raster (`.tif`). The resulting rasters will have 2 bands, one for each statistic we calculate based on the vector geometries in each pixel.
![Ice wedge polygons at high resolution, visible when zoomed-in to the Permafrost Discovery Gateway Imagery Viewer.](../images/zoom_in_IWP.png)

We need to create the highest zoom level rasters before we create the lower zoom levels in the next lesson. This brings us one step closer to visualizing the ice wedge polygons on a basemap, with the ability to zoom in and out!
The higher resolution tiles are the "child" tiles, and the lower resolution tiles are the "parent" tiles.

### Packages, Libraries, and Modules

- [os 3.11.2](https://docs.python.org/3/library/os.html)
- [parsl 2023.3.20](https://parsl.readthedocs.io/en/stable/)
- [pdgstaging](https://github.com/PermafrostDiscoveryGateway/viz-staging)
- package developed by Permafrost Discovery Gateway software developer and designer Robyn Thiessen-Bock
- developed by Permafrost Discovery Gateway software developer and designer Robyn Thiessen-Bock
- [pdgraster](https://github.com/PermafrostDiscoveryGateway/viz-raster/tree/main)
- package developed by Permafrost Discovery Gateway software developer and designer Robyn Thiessen-Bock
- developed by Permafrost Discovery Gateway software developer and designer Robyn Thiessen-Bock
- [geopandas0.11](https://geopandas.org/en/stable/docs.html)
- [random](https://docs.python.org/3/library/random.html)
- [matplotlib 3.5](https://matplotlib.org/3.5.3/api/_as_gen/matplotlib.pyplot.html)
- [matplotlib 3.5](https://matplotlib.org/3.5.3/api/_as_gen/matplotlib.pyplot.html)
- [ipyleaflet 0.17](https://ipyleaflet.readthedocs.io/en/latest/)
45 changes: 0 additions & 45 deletions sections/group-project-3.qmd

This file was deleted.

0 comments on commit 6112a35

Please sign in to comment.