EDR CoverageJSON conformance fixes: time axis and demo data #1814

pzaborowski · 2024-09-20T09:20:21Z

Overview

changes to confirm to https://schemas.opengis.net/covjson/1.0/coveragejson.json

time axis changed to values (was interval)
time axis first in range as expected by https://github.com/Reading-eScience-Centre/leaflet-coverage and dependancies
parameter description as object - worked with JS visualisation but not validator and covjson playground
range dataType as float (was number and allowed are float, integer, string) - worked with JS visualisation but not validator and covjson playground

Other:

typo in edr/query.html template causing issues with first parameter layer
- Changes from Edr fixed axis #1811 included

Changes shall solve issues using https://covjson.org libraries, also on pygeoapi demo site.

Related Issue / discussion

mainly #1833
but pull depends on this
#1782

Additional information

changes is not fully backward compatible with potential non-standard client using time as interval (which is breaking coveragejson-schema).

In addition to code changes, coads_sst.nc is fixed. Demo file had COADSX shifted west but not data which looks like this:

Dependency policy (RFC2)

I have ensured that this PR meets RFC2 requirements

Updates to public demo

I have ensured that breaking changes to the pygeoapi master demo server have been addressed
- https://github.com/geopython/demo.pygeoapi.io/blob/master/services/pygeoapi_master/local.config.yml

Contributions and licensing

(as per https://github.com/geopython/pygeoapi/blob/master/CONTRIBUTING.md#contributions-and-licensing)

I'd like to contribute [feature X|bugfix Y|docs|something else] to pygeoapi. I confirm that my contributions to pygeoapi will be compatible with the pygeoapi license guidelines at the time of contribution
I have already previously agreed to the pygeoapi Contributions and Licensing Guidelines

Reading from query bbox and reversing the order would not work anyway if not followed by reversing data (which is not cheap and would not do on default. It is visible on the sample data tests/data/coads_sst.nc where the start-stop swapping turn the data upside down.

there is no sense in reading it from source file once again

Edr fixed axis

1. time axis shall be 't' in covjson, for backward compliance it can be set with 'time_axis_covjson' provider variable. if time_axis_covjson is not set the value is read from time_field which has default value read from the source file. 2. time axis is now list of values according to specification and it is now properly rendered on the html view of the query 3. typo in query.html to fix error with first layer being not generated

1.integrated changes from axis fix 2. dataType is float according to schema https://schemas.opengis.net/covjson/1.0/coveragejson.json (number is not valid, float, integer, string are) 3. parameter description as object according to schema https://schemas.opengis.net/covjson/1.0/coveragejson.json

coards_sst.nc data was shifted around 180 deg

Edr fixed axis

* Update compliance for OGC API - Processes * Update introduction.rst

* fix queryables provider handling * fix test

* Manage non-cf-compliant time dimension * Manage datasets without a time dimension * Allow reversed slices also for axes * Convert also metadata to float64 for json output * Use named temporary file to enable netcdf4 engine * Make float64 conversion faster * Add netcdf output to xarray provider * Flake8 fixes * Fix bug when no time axis in data * Use new xarray interface * Add test for zarr dataset without time dimension * Avoid errors if missing long_name * Manage zarr and netcdf output in the same way * Revert "Manage zarr and netcdf output in the same way" This reverts commit 0b09281. * Revert "Add netcdf output to xarray provider" This reverts commit 9f72bf7.

* Added ability for self-hosted token service to be specified. * Update documentation to show the available parameters * Update pygeoapi/provider/esri.py Co-authored-by: Benjamin Webb <[email protected]> * Update pygeoapi/provider/esri.py Co-authored-by: Benjamin Webb <[email protected]> * Update pygeoapi/provider/esri.py Co-authored-by: Benjamin Webb <[email protected]> * Update pygeoapi/provider/esri.py Co-authored-by: Benjamin Webb <[email protected]> * Update pygeoapi/provider/esri.py * Update ogcapi-features.rst --------- Co-authored-by: Benjamin Webb <[email protected]> Co-authored-by: Tom Kralidis <[email protected]>

… need to use the MemoryFile. (geopython#1824)

webb-ben

Looks good. Please rebase!

webb-ben · 2024-10-24T18:48:23Z

tests/api/test_environmental_data_retrieval.py

+    # assert time_dict['start'] == '2000-08-16T07:23:42.000000000'
+    # assert time_dict['stop'] == '2000-12-16T01:20:05.999999996'
+    # assert time_dict['num'] == 7


Suggested change

# assert time_dict['start'] == '2000-08-16T07:23:42.000000000'

# assert time_dict['stop'] == '2000-12-16T01:20:05.999999996'

# assert time_dict['num'] == 7

webb-ben · 2024-10-24T18:49:31Z

pygeoapi/templates/collections/edr/query.html

This should be already addressed in upstream pygeoapi

webb-ben · 2024-10-24T18:50:55Z

tests/api/test_environmental_data_retrieval.py

+    # assert time_dict['start'] == '2000-06-16T10:25:30.000000000'
+    # assert time_dict['stop'] == '2000-08-16T07:23:42.000000000'
+    # assert time_dict['num'] == 3


Suggested change

# assert time_dict['start'] == '2000-06-16T10:25:30.000000000'

# assert time_dict['stop'] == '2000-08-16T07:23:42.000000000'

# assert time_dict['num'] == 3

webb-ben · 2024-10-24T18:51:00Z

tests/api/test_environmental_data_retrieval.py

+    # assert time_dict['start'] == '2000-06-16T10:25:30.000000000'
+    # assert time_dict['stop'] == '2000-08-16T07:23:42.000000000'
+    # assert time_dict['num'] == 3


Suggested change

# assert time_dict['start'] == '2000-06-16T10:25:30.000000000'

# assert time_dict['stop'] == '2000-08-16T07:23:42.000000000'

# assert time_dict['num'] == 3

webb-ben · 2024-10-24T18:51:23Z

pygeoapi/provider/xarray_.py

+            self.time_axis_covjson = provider_def.get('time_axis_covjson') \
+                or self.time_field


What is the point of adding this? Can we add a test case that uses this field. Suggest to potentially rename in pygeoapi, but will defer to @tomkralidis. This addition should also be noted in the docs (OACoverages and OAEDR

time dimension in configuration just for backward compatibility, it is expected in covjson to be 't'

pzaborowski and others added 25 commits August 12, 2024 11:32

clean axis y order

5cf2ee3

Reading from query bbox and reversing the order would not work anyway if not followed by reversing data (which is not cheap and would not do on default. It is visible on the sample data tests/data/coads_sst.nc where the start-stop swapping turn the data upside down.

remove comments

1b5432c

fixed axis from data/bbox

3fc3b5f

bbox read from already calculated metadata

2feee54

there is no sense in reading it from source file once again

Merge branch 'upsorg' into edr_fixed_axis

d583d32

fix

f0c2dd0

whitespaces

d3c6726

Merge pull request #11 from pzaborowski/edr_fixed_axis

dfce634

Edr fixed axis

Update coads_sst.nc

8531e40

coards_sst.nc data was shifted around 180 deg

Merge branch 'EDR-time-axis-values' into edr_fixed_axis

baf9c7c

formatting

9c0be8b

Merge pull request #12 from pzaborowski/edr_fixed_axis

57fb90f

Edr fixed axis

Update Ubuntu Jammy docker base image to 20240911.1 (geopython#1815)

76fd130

docs: update compliance for OGC API - Processes (geopython#1817)

52bec0f

* Update compliance for OGC API - Processes * Update introduction.rst

Zoom to first layer on EDR (geopython#1819)

6b91024

update release version

83ef1ac

back to dev

b3a7071

fix item queryables provider handling (geopython#1820)

474cb60

* fix queryables provider handling * fix test

Fixes a memoryfile issue with rasterio. When using f=json, it doesn't…

179c90f

… need to use the MemoryFile. (geopython#1824)

Properly support int variables of any width (geopython#1829)

3bdeefe

tests aligned with data, unneccesary time properties removed

c38ad33

pzaborowski mentioned this pull request Oct 23, 2024

EDR API compliance with CoverageJSON and tools #1833

Open

pzaborowski added 3 commits October 23, 2024 17:01

Merge branch 'upsorg'

368fc34

Merge branch 'edr_fixed_axis'

1431963

Merge branch 'EDR-time-axis-values'

e6436eb

tomkralidis requested a review from webb-ben October 24, 2024 17:42

tomkralidis self-requested a review October 24, 2024 17:42

tomkralidis added the bug Something isn't working label Oct 24, 2024

tomkralidis added this to the 0.19.0 milestone Oct 24, 2024

webb-ben reviewed Oct 24, 2024

View reviewed changes

time dimension as first and with label from configuration

12f9c5d

time dimension in configuration just for backward compatibility, it is expected in covjson to be 't'

tomkralidis modified the milestones: 0.19.0, 0.20.0 Dec 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

EDR CoverageJSON conformance fixes: time axis and demo data #1814

EDR CoverageJSON conformance fixes: time axis and demo data #1814

pzaborowski commented Sep 20, 2024 •

edited

Loading

webb-ben left a comment

webb-ben Oct 24, 2024

webb-ben Oct 24, 2024

webb-ben Oct 24, 2024

webb-ben Oct 24, 2024

webb-ben Oct 24, 2024

	# assert time_dict['start'] == '2000-08-16T07:23:42.000000000'
	# assert time_dict['stop'] == '2000-12-16T01:20:05.999999996'
	# assert time_dict['num'] == 7

	# assert time_dict['start'] == '2000-06-16T10:25:30.000000000'
	# assert time_dict['stop'] == '2000-08-16T07:23:42.000000000'
	# assert time_dict['num'] == 3

		self.time_axis_covjson = provider_def.get('time_axis_covjson') \
		or self.time_field

EDR CoverageJSON conformance fixes: time axis and demo data #1814

Are you sure you want to change the base?

EDR CoverageJSON conformance fixes: time axis and demo data #1814

Conversation

pzaborowski commented Sep 20, 2024 • edited Loading

Overview

Related Issue / discussion

Additional information

Dependency policy (RFC2)

Updates to public demo

Contributions and licensing

webb-ben left a comment

Choose a reason for hiding this comment

webb-ben Oct 24, 2024

Choose a reason for hiding this comment

webb-ben Oct 24, 2024

Choose a reason for hiding this comment

webb-ben Oct 24, 2024

Choose a reason for hiding this comment

webb-ben Oct 24, 2024

Choose a reason for hiding this comment

webb-ben Oct 24, 2024

Choose a reason for hiding this comment

pzaborowski commented Sep 20, 2024 •

edited

Loading