Add OGC Features API at /catalogs/{catalogId}/collections/{collection…

…Id}/features (#265)

* Change routes under /catalogs/{catalogId} to be implemented as a separate application such that it has its own docs and can function as a standalone OGC Features API

* Initial object endpoints

* endpoints landing page, conformance, collections and collections/{collectionId} working

* Integrate rdf2geojson

* Dynamic /items query to integrate CQL

* Update feature query to "that make the Observations on the Feature optional (so it can render Sites, FoIs, and other non-Occurrence geo:Features we might have"

* Queryables WIP

* Add queryables endpoints

* Add bbox query param

* Incorporate call to iri2curie function w/ geojson to utilise curies as identifiers rather than iris.

* Fix ns1 ns2 issues; add namespaces to returned responses always

* Re-integrate OGC features w/ vanilla prez
Update profile to make props separately optional
Add link headers to JSON responses

* Tidy up mediatype specification; ensure RDF mediatypes can be returned from OGC Features endpoints

* Change remote endpoint / sparql query template to SELECT query so pyoxigraph his supported. Remove BDR specific query.
Optimise imports. Fix 4*tests

* Add endpoint to show prez prefixes

* Add /prefixes endpoint. Start updating query params references to use enums.

* Update python to 3.12 in github actions

* Remove BDR specific files/config

* Update documentation with new API structure/configuration.

* Black all code.

* Remove commented code.

* slim deps

* Implement full CQL time functions

* Update poetry.lock and black

* Add uvicorn server dep to github action

* Fix invalid bbox format for Geomet
OGC Features compliance changes

* Remove OGC Features specific Dockerfile, move changes to "base" Dockerfile

* Add zeroOrMorePath, oneOrMorePath, zeroOrOnePath SHACL. Convert property selection to UNION queries.

* fix: update ogc_tests so they can run properly.

* Uvicorn is imported in a try/except block as an optional dependency. OGC Features can be disabled/enabled. Enabled by default.

* Parameterise OGC tests and xfail the CRS & api Definition tests.
Black all code.

* fix: relative path issue

couldn't get pytest to run from the command line

* Fix broken tests by clarifying which test data goes with SpacePrez/OGC Features, add geometries to SpacePrez data so it renders. Add links to display of a single feature.

---------

Co-authored-by: Lawon Lewis <[email protected]>

.github/workflows/on_pr_to_main.yaml

@@ -22,7 +22,7 @@ jobs:
         id: setup-python
         uses: actions/setup-python@v4
         with:
-          python-version: '3.11'
+          python-version: '3.12'
       #----------------------------------------------
       #  -----  install & configure poetry  -----
       #----------------------------------------------
@@ -52,7 +52,7 @@ jobs:
       # install your root project, if required
       #----------------------------------------------
       - name: Install project
-        run: poetry install --no-interaction
+        run: poetry install --no-interaction --extras "server"
       #----------------------------------------------
       #              run test suite
       #----------------------------------------------

.github/workflows/on_push_to_feature.yaml

@@ -20,7 +20,7 @@ jobs:
         id: setup-python
         uses: actions/setup-python@v4
         with:
-          python-version: '3.11'
+          python-version: '3.12'
       #----------------------------------------------
       #  -----  install & configure poetry  -----
       #----------------------------------------------
@@ -50,7 +50,7 @@ jobs:
       # install your root project, if required
       #----------------------------------------------
       - name: Install project
-        run: poetry install --no-interaction
+        run: poetry install --no-interaction --extras "server"
       #----------------------------------------------
       #              run test suite
       #----------------------------------------------

.github/workflows/on_release.yaml

@@ -28,7 +28,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: "3.11"
+          python-version: "3.12"
       - name: Install Poetry
         uses: snok/install-poetry@v1
 

Dockerfile

@@ -15,13 +15,8 @@ ENV VIRTUAL_ENV=${VIRTUAL_ENV} \
 
 RUN apk add --no-cache \
       bash \
-      gcc \
-      libffi-dev \
-      musl-dev \
       pipx \
-      python3-dev \
-      geos \
-      geos-dev
+    git
 
 RUN pipx install poetry==${POETRY_VERSION}
 
@@ -32,6 +27,7 @@ COPY . .
 RUN poetry build
 RUN python3 -m venv --system-site-packages ${VIRTUAL_ENV}
 RUN ${VIRTUAL_ENV}/bin/pip3 install --no-cache-dir dist/*.whl
+RUN ${VIRTUAL_ENV}/bin/pip3 install uvicorn
 
 #
 # Final
@@ -48,10 +44,8 @@ COPY --from=base ${VIRTUAL_ENV} ${VIRTUAL_ENV}
 
 RUN apk update && \
     apk upgrade --no-cache && \
-    apk add --no-cache \
-      bash \
-      geos \
-      geos-dev
+    apk add --no-cache
+
 
 WORKDIR /app
 # prez module is already built as a package and installed in $VIRTUAL_ENV as a library

README-Dev.md

@@ -561,6 +561,42 @@ these endpoints, specifying any variables that need to be substituted (such as p
 to construct the system links.
 5. Return the response
 
+## Specification of remote SPARQL templates for Object endpoints
+
+The OGC Features endpoints can utilise custom SPARQL templates.
+At present the queries must be of the form:
+```sparql
+PREFIX geo: <http://www.opengis.net/ont/geosparql#>
+PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
+PREFIX sosa: <http://www.w3.org/ns/sosa/>
+CONSTRUCT {
+ ...
+}
+WHERE {
+    VALUES ?focusNode { UNDEF } 
+...
+}
+```
+At present the queries are ONLY substituted with the focus node URI, and only for OGC Features endpoints. It is intended that this functionality will provide the basis for a more general templating system across all OBJECT endpoints in the future.  
+These templates should be declared in the remote repo in this format:  
+```sparql
+PREFIX prez: <https://prez.dev/ont/>
+PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
+INSERT DATA { GRAPH <https://prez/system> {
+    [ a prez:TemplateQuery ;
+        rdf:value """<template_query>
+""" ;
+        prez:forEndpoint "http://www.opengis.net/ogcapi-features-1/1.0/feature" ;
+    ]
+    }}
+```
+Prez will detect these template queries and when a request comes in on the endpoint with the relevant URI, utilise the template query.
+
+Templates can also be specified in `prez/reference_data/xxx.rq`, and mapped to an endpoint using the `endpoint_to_template_query_filename` setting (can be set as an environment variable), where `xxx.rq` is the filename of the template query. For example:
+```bash
+export ENDPOINT_TO_TEMPLATE_QUERY_FILENAME='{"http://www.opengis.net/ogcapi-features-1/1.0/feature": "xxx.rq"}'
+```
+
 ## High Level Sequence listing and individual object endpoints
 
 Prez follows the following logic to determine what information to return, based on a profile, and in what mediatype to return it.

README.md

@@ -40,6 +40,50 @@ poetry install
 
 > **Note:** Poetry must be installed on the system. To check if you have Poetry installed run `poetry --version`. For tips on installing and managing specific dependency groups check the [documentation](https://python-poetry.org/docs/managing-dependencies/).
 
+## Endpoints
+
+Prez delivers the following endpoints:
+
+### Core Endpoints
+
+**Endpoint** | **Default MT**
+--- | ---
+/ | text/anot+turtle
+/docs | text/html
+/catalogs/{catalogId} | text/anot+turtle
+/catalogs/{catalogId}/collections | text/anot+turtle
+/catalogs/{catalogId}/collections/{recordsCollectionId} | text/anot+turtle
+/catalogs/{catalogId}/collections/{recordsCollectionId}/items | text/anot+turtle
+/catalogs/{catalogId}/collections/{recordsCollectionId}/items/{itemId} | text/anot+turtle
+/purge-tbox-cache | application/json
+/tbox-cache | application/json
+/health | application/json
+/prefixes | text/anot+turtle
+/concept-hierarchy/{parent_curie}/narrowers | text/anot+turtle
+/concept-hierarchy/{parent_curie}/top-concepts | text/anot+turtle
+/cql | text/anot+turtle
+/profiles | text/anot+turtle
+/search | text/anot+turtle
+/profiles/{profile_curie} | text/anot+turtle
+/object | text/anot+turtle
+/identifier/redirect | N/A
+/identifier/curie/{iri} | text/plain
+/identifier/iri/{curie} | text/plai
+
+### OGC Features API Endpoints
+
+**The OGC Features API Endpoints are based at the ROOT `/catalogs/{catalogId}/collections/{recordsCollectionId}/`**
+
+**Endpoint** | **Default MT**
+--- | ---
+{ROOT}/features | application/json
+{ROOT}/features/docs | text/html
+{ROOT}/features/conformance | application/json
+{ROOT}/features/collections | application/json
+{ROOT}/features/collections/{collectionId} | application/json
+{ROOT}/features/collections/{collectionId}/items | application/geo+json
+{ROOT}/features/collections/{collectionId}/items/{featureId} | application/geo+jso
+
 ## Configuration
 
 The following Environment Variables can be used to configure Prez:
@@ -144,6 +188,23 @@ Used in conjunction with the Pyoxigraph repo. Specifies a directory (from the re
     - `ep:system/profile-listing`
     - `ep:system/profile-object`
 
+#### Listing and Search Configuration
+
+- **`listing_count_limit`**: The maximum number of items to count for a listing endpoint. Counts greater than this limit will be returned as ">N" where N is the limit. Default is `100`.
+- **`search_count_limit`**: The maximum number of items to return in a search result. Default is `10`.
+
+#### SPARQL Endpoint
+
+- **`enable_sparql_endpoint`**: Whether to enable the SPARQL endpoint. Default is `False`.
+
+#### Temporal Configuration
+
+- **`temporal_predicate`**: The predicate used for temporal properties. Default is `sdo:temporal`.
+
+#### Query Template Configuration
+
+- **`endpoint_to_template_query_filename`**: A dictionary mapping endpoints to query template filenames. Default is an empty dictionary.
+
 ## Running
 
 This section is for developing Prez locally. See the [Running](#running) options below for running Prez in production.

azure/function_app.py

@@ -59,4 +59,3 @@
     task = fn(req, context)
     resp = loop.run_until_complete(task)
     print(resp)
-

azure/patched_asgi_function_wrapper.py

@@ -7,6 +7,7 @@
 from azure.functions._abc import Context
 from azure.functions import HttpRequest
 
+
 # -------------------
 # Create a patched AsgiFunctionApp to fix the ASGI scope state issue
 # -------------------
@@ -16,12 +17,13 @@ async def _handle_async(self, req, context):
         asgi_request = AsgiRequest(req, context)
         scope = asgi_request.to_asgi_http_scope()
         # shallow copy the state as-per the ASGI spec
-        scope["state"] = copy(self.state)  # <-- this is the patch, add the state to the scope
-        asgi_response = await AsgiResponse.from_app(self._app,
-                                                    scope,
-                                                    req.get_body())
+        scope["state"] = copy(
+            self.state
+        )  # <-- this is the patch, add the state to the scope
+        asgi_response = await AsgiResponse.from_app(self._app, scope, req.get_body())
         return asgi_response.to_func_response()
 
+
 # -------------------
 # Create a patched AsgiFunctionApp to fix the double-slash route issue
 # -------------------
@@ -35,7 +37,7 @@ def __init__(self, app, http_auth_level):
         self.startup_task_done = False
 
     def _add_http_app(
-            self, http_middleware: Union[AsgiMiddleware, WsgiMiddleware]
+        self, http_middleware: Union[AsgiMiddleware, WsgiMiddleware]
     ) -> None:
         """Add an Asgi app integrated http function.