Skip to content

Commit

Permalink
feat: work in progress, replace pure python --> PostGIS functions
Browse files Browse the repository at this point in the history
  • Loading branch information
spwoodcock committed Jan 9, 2025
1 parent 2ce0441 commit 3dc30ff
Show file tree
Hide file tree
Showing 13 changed files with 377 additions and 431 deletions.
7 changes: 7 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ RUN apt-get update --quiet \
apt-get install -y --quiet --no-install-recommends \
"build-essential" \
"gcc" \
"libpq-dev" \
&& rm -rf /var/lib/apt/lists/*
COPY --from=uv /uv /usr/local/bin/uv
COPY pyproject.toml uv.lock README.md /_lock/
Expand Down Expand Up @@ -103,6 +104,12 @@ ENV PYTHONDONTWRITEBYTECODE=1 \
SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt \
REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt \
CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
RUN apt-get update --quiet \
&& DEBIAN_FRONTEND=noninteractive \
apt-get install -y --quiet --no-install-recommends \
"postgresql-client" \
&& DEBIAN_FRONTEND=noninteractive apt-get upgrade -y \
&& rm -rf /var/lib/apt/lists/*
# Copy Python deps from build to runtime
COPY --from=build /opt/python /opt/python
WORKDIR /data
Expand Down
29 changes: 16 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

<!-- markdownlint-disable -->
<p align="center">
<img src="https://github.com/hotosm/fmtm/blob/main/images/hot_logo.png?raw=true" style="width: 200px;" alt="HOT"></a>
<img src="https://raw.githubusercontent.com/hotosm/geojson-aoi-parser/refs/heads/main/docs/images/hot_logo.png" style="width: 200px;" alt="HOT"></a>
</p>
<p align="center">
<em>Parse and normalize a GeoJSON area of interest, using pure Python.</em>
<em>Parse and normalize a GeoJSON area of interest, using using PostGIS.</em>
</p>
<p align="center">
<a href="https://github.com/hotosm/geojson-aoi-parser/actions/workflows/docs.yml" target="_blank">
Expand Down Expand Up @@ -54,25 +54,16 @@
- **Flexible geometry input**:
- Polygon
- MultiPolygons
- GeometryCollection
- Feature
- FeatureCollection
- Handle multigeometries with an optional merge to single polygon, or split into
featcol of individual polygons.
- Handle geometries nested inside GeometryCollection*.
- Handle geometries nested inside GeometryCollection.
- Remove any z-dimension coordinates.
- Warn user if CRS is provided, in a coordinate system other than EPSG:4326.
- **Normalised output**: FeatureCollection containing Polygon geoms.

> [!WARNING]
> *We typically advise against using the GeometryCollection type, and support
> in this library may not be fully functional.
>
> However sometimes geometries may need to be returned wrapped in
> GeometryCollection, for example due to idiosyncrasies of PostGIS.
>
> In this scenario, we support stripping out the first geometry from inside
> each GeometryCollection object (that may be nested in a FeatureCollection).
## Capturing The Warnings

If the GeoJSON has an invalid CRS, or coordinates seem off, a warning
Expand Down Expand Up @@ -104,3 +95,15 @@ if recorded_warnings:
# do stuff with warning
logger.warning(f"A warning was encountered: {warning.message}")
```

## History

- Initially I tried to write a pure-Python implementation of this, no dependencies.
- I underestimated the amount of work that is! It could be possible to reverse
engineer C++ Geos or georust/geos, but it's more hassle than it's worth.
- As all of the target install candidates for this package use a db driver
anyway, I thought it wisest (and most time efficient) to use the PostGIS
Geos implementation (specifically for the unary_union and convex_hull
algorithms).
- An additional advantage is the potential to port this to PGLite when the
PostGIS extension is available, meaning AOI processing easily in the browser.
33 changes: 30 additions & 3 deletions compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,12 @@
# along with geojson-aoi-parser. If not, see <https:#www.gnu.org/licenses/>.
#

networks:
net:
name: aoi-parser

services:
aoi-parser:
parser:
image: "ghcr.io/hotosm/geojson-aoi-parser:${TAG_OVERRIDE:-ci}"
build:
target: ci
Expand All @@ -26,6 +30,29 @@ services:
- ./geojson_aoi:/opt/python/lib/python3.10/site-packages/geojson_aoi
# Mount local tests
- ./tests:/data/tests
network_mode: none
restart: "unless-stopped"
depends_on:
db:
condition: service_healthy
networks:
- net
restart: "no"
command: "pytest"

db:
image: "postgis/postgis:17-3.5-alpine"
container_name: aoi-parser-db
environment:
- POSTGRES_USER=aoi
- POSTGRES_PASSWORD=dummycipassword
- POSTGRES_DB=aoi
ports:
- "5439:5432"
networks:
- net
restart: "unless-stopped"
healthcheck:
test: pg_isready -U ${FMTM_DB_USER:-aoi} -d ${FMTM_DB_NAME:-aoi}
start_period: 5s
interval: 10s
timeout: 5s
retries: 3
182 changes: 0 additions & 182 deletions geojson_aoi/merge.py

This file was deleted.

Loading

0 comments on commit 3dc30ff

Please sign in to comment.