feat(basemaps): New cli listing jobs for topo raster map standardised workflow. BM-1127

[Wentao-Kuang]

This description is a WIP.

---

Motivation

As a Basemaps user, I want to consume the NZTopo 50 & 250 Maps as a tile service.

Modifications

This work defines a new Argo task (CLI listing). This task's purpose is to generate STAC files for an NZTopo Map Series imagery collection. We have designed this task to process the following collections stored in the AWS S3 TopoReleaseArchive directory:

s3://topographic-upload/TopoReleaseArchive/NZTopo50_GeoTif_Gridless/
s3://topographic-upload/TopoReleaseArchive/NZTopo250_GeoTif_Gridless/

There is also a new Argo workflow in development that depends on this work. The workflow's purpose is to automate the standardisation of an NZTopo Map Series imagery collection. The workflow executes this task as a step in its process flow.

Argo task: topo-stac-creation

Arguments

The arguments that can be passed to the task are as follows:

• Title

  The name/title of the Map Series imagery collection.

  Argument	title
  Type	string
  Required	yes
  Example	Raster Topographic Maps 50k

• Source

  The source directory URL of the Map Series imagery collection.

  Argument	source
  Type	string
  Required	yes
  Example	s3://topographic-upload/TopoReleaseArchive/NZTopo50_GeoTif_Gridless/

• Target

  The target directory URL into which to save the generated directory structure of StacItem and StacCollection files.

  Argument	target
  Type	string
  Required	yes
  Example	s3://linz-workflows-scratch/<date>/<hash>/

• Scale

  The scale of the Map Series imagery collection's map sheets.

  Argument	scale
  Type	string
  Required	yes
  Options	topo25, topo50, or topo250

• Resolution

  The resolution of the Map Series imagery collection's map sheets.

  Argument	resolution
  Type	string
  Required	yes
  Options	gridded_600dpi or gridless_600dpi

• Latest Only

  A flag used to indicate whether all of the generated files should be saved to the target location, or only that of each map sheet's latest version.

  Argument	latestOnly
  Type	boolean
  Required	no
  Options	true or false

Process

The task processes a collection as follows:

1. Loads the collection of GeoTiff images into memory
2. Extracts the map code, version, and metadata (e.g. EPSG) from each image
3. Identifies the latest version of each map sheet by code
4. Groups the images by EPSG
5. Generates a StacItem file for each image, and a StacCollection file for each EPSG grouping
6. Structures the groups of StacItem and StacCollection files into a directory tree
7. Saves the files as they are structured to a target location

Outputs

The task groups the images by EPSG and then structures the generated StacItem and StacCollection files, as illustrated:

Schema	Example

The task then saves the generated tree of folders and files into the target location directory.

Limitations

• Tile Matrices

  For each image in a collection, the task extracts the EPSG of the image and converts it to a runtime Epsg Enum value. This value is then mapped to the EPSG's corresponding Tile Matrix definition. At this time, there is no such Tile Matrix definition for the Chatham Islands EPSG code, 3793. The task will need to be updated once Basemaps supports this definition.

---

Checklist

Local validation

• Tests updated - Local tests
• Docs updated - Just want to merge this for testing in workflow for now, to make the document later once got another cli for tagging topo raster versions.
• Issue linked in Title

[tawera-manaena]

I'm revising some of the documentation, some of which describe earlier ways in which we did things. I'll make sure all of the docs are consistent before marking this pull request as "ready for review". Won't take me long.