Merge branch 'master' into add-pre-commit-hook-vulture

.github/linters/.markdown-lint.yml

@@ -30,17 +30,11 @@ MD034: false
 # no-emphasis-as-heading - Emphasis used instead of a heading
 MD036: false
 
-# no-space-in-code - Spaces inside code span elements
-MD038: false
-
 # fenced-code-language - Fenced code blocks should have a language specified
 MD040: false
 
 # first-line-heading/first-line-h1 - First line in a file should be a top-level heading
 MD041: false
 
-# no-alt-text - Images should have alternate text (alt text)
-MD045: false
-
 # code-block-style - Code block style
 MD046: false

.github/linters/.yaml-lint.yml

@@ -4,8 +4,6 @@ extends: default
 
 rules:
   colons: disable
-  comments: disable
-  comments-indentation: disable
   document-start: disable
   line-length: disable
   truthy: false

.github/pull_request_template.md

@@ -10,7 +10,9 @@
 
 - Yes, the URL of the associated JIRA ticket is https://issues.apache.org/jira/browse/SEDONA-XXX. The PR name follows the format `[SEDONA-XXX] my subject`.
 
-- No, this is a documentation update. The PR name follows the format `[DOCS] my subject`.
+- No:
+  - this is a documentation update. The PR name follows the format `[DOCS] my subject`
+  - this is a CI update. The PR name follows the format `[CI] my subject`
 
 
 ## What changes were proposed in this PR?

.pre-commit-config.yaml

@@ -18,6 +18,16 @@ repos:
     rev: 24.10.0
     hooks:
       - id: black-jupyter
+  - repo: https://github.com/pycqa/isort
+    rev: 5.13.2
+    hooks:
+      - id: isort
+        name: isort (python)
+  - repo: https://github.com/PyCQA/bandit
+    rev: 1.7.10
+    hooks:
+      - id: bandit
+        args: ["-c=pyproject.toml", "-r"]
   - repo: https://github.com/codespell-project/codespell
     rev: v2.3.0
     hooks:
@@ -30,6 +40,11 @@ repos:
     rev: v8.20.1
     hooks:
       - id: gitleaks
+  - repo: https://github.com/shssoichiro/oxipng
+    rev: v9.1.2
+    hooks:
+      - id: oxipng
+        args: ["-o", "4", "--strip", "safe", "--alpha"]
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v5.0.0
     hooks:

R/README.md

@@ -1,4 +1,4 @@
-# apache.sedona <img src="man/figures/logo.png" align="right" width="120"/>
+# apache.sedona <img src="man/figures/logo.png" align="right" alt="Apache Sedona Logo" width="120"/>
 
 [Apache Sedona](https://sedona.apache.org/) is a cluster computing system for processing large-scale spatial data. Sedona extends existing cluster computing systems, such as Apache Spark and Apache Flink, with a set of out-of-the-box distributed Spatial Datasets and Spatial SQL that efficiently load, process, and analyze large-scale spatial data across machines.
 

R/_pkgdown.yml

@@ -3,7 +3,7 @@ template:
   bootstrap: 5
   # bootswatch: cosmo
   bslib:
-    #font_scale: 1.1
+    # font_scale: 1.1
     primary: "#ff6e42"
     light: "#ff6e42"
     navbar-light-color: "#fff"
@@ -13,7 +13,7 @@ template:
     # bg: "#fff"
 navbar:
   structure:
-   # left:  [intro, articles, reference, news]
+    # left: [intro, articles, reference, news]
     right: [search, github]
 home:
   sidebar:

README.md

@@ -2,10 +2,10 @@
 
 [![Scala and Java build](https://github.com/apache/sedona/actions/workflows/java.yml/badge.svg)](https://github.com/apache/sedona/actions/workflows/java.yml) [![Python build](https://github.com/apache/sedona/actions/workflows/python.yml/badge.svg)](https://github.com/apache/sedona/actions/workflows/python.yml) [![R build](https://github.com/apache/sedona/actions/workflows/r.yml/badge.svg)](https://github.com/apache/sedona/actions/workflows/r.yml) [![Docker image build](https://github.com/apache/sedona/actions/workflows/docker-build.yml/badge.svg)](https://github.com/apache/sedona/actions/workflows/docker-build.yml) [![Example project build](https://github.com/apache/sedona/actions/workflows/example.yml/badge.svg)](https://github.com/apache/sedona/actions/workflows/example.yml) [![Docs build](https://github.com/apache/sedona/actions/workflows/docs.yml/badge.svg)](https://github.com/apache/sedona/actions/workflows/docs.yml)
 
-| Download statistics        | **Maven**  | **PyPI**                                                                                                                                                                                                                                                                                                                                     | Conda-forge                                                                                                                                     | **CRAN**                                                                                                                                                                                                                                                          | **DockerHub**                                                                                                                  |
-|----------------------------|------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------|
-| Apache Sedona              | 225k/month | [![PyPI - Downloads](https://img.shields.io/pypi/dm/apache-sedona)](https://pepy.tech/project/apache-sedona) [![Downloads](https://static.pepy.tech/personalized-badge/apache-sedona?period=total&units=international_system&left_color=black&right_color=brightgreen&left_text=total%20downloads)](https://pepy.tech/project/apache-sedona) | [![Anaconda-Server Badge](https://anaconda.org/conda-forge/apache-sedona/badges/downloads.svg)](https://anaconda.org/conda-forge/apache-sedona) | [![](https://cranlogs.r-pkg.org/badges/apache.sedona?color=brightgreen)](https://cran.r-project.org/package=apache.sedona) [![](https://cranlogs.r-pkg.org/badges/grand-total/apache.sedona?color=brightgreen)](https://cran.r-project.org/package=apache.sedona) | [![Docker pulls](https://img.shields.io/docker/pulls/apache/sedona?color=brightgreen)](https://hub.docker.com/r/apache/sedona) |
-| Archived GeoSpark releases | 10k/month  | [![PyPI - Downloads](https://img.shields.io/pypi/dm/geospark)](https://pepy.tech/project/geospark)[![Downloads](https://static.pepy.tech/personalized-badge/geospark?period=total&units=international_system&left_color=black&right_color=brightgreen&left_text=total%20downloads)](https://pepy.tech/project/geospark)                      |                                                                                                                                                 |                                                                                                                                                                                                                                                                   |                                                                                                                                |
+| Download statistics        | **Maven**  | **PyPI**                                                                                                                                                                                                                                                                                                                                     | Conda-forge                                                                                                                                     | **CRAN**                                                                                                                                                                                                                                                                                                    | **DockerHub**                                                                                                                  |
+|----------------------------|------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------|
+| Apache Sedona              | 225k/month | [![PyPI - Downloads](https://img.shields.io/pypi/dm/apache-sedona)](https://pepy.tech/project/apache-sedona) [![Downloads](https://static.pepy.tech/personalized-badge/apache-sedona?period=total&units=international_system&left_color=black&right_color=brightgreen&left_text=total%20downloads)](https://pepy.tech/project/apache-sedona) | [![Anaconda-Server Badge](https://anaconda.org/conda-forge/apache-sedona/badges/downloads.svg)](https://anaconda.org/conda-forge/apache-sedona) | [![CRAN monthly downloads](https://cranlogs.r-pkg.org/badges/apache.sedona?color=brightgreen)](https://cran.r-project.org/package=apache.sedona) [![Total CRAN downloads](https://cranlogs.r-pkg.org/badges/grand-total/apache.sedona?color=brightgreen)](https://cran.r-project.org/package=apache.sedona) | [![Docker pulls](https://img.shields.io/docker/pulls/apache/sedona?color=brightgreen)](https://hub.docker.com/r/apache/sedona) |
+| Archived GeoSpark releases | 10k/month  | [![PyPI - Downloads](https://img.shields.io/pypi/dm/geospark)](https://pepy.tech/project/geospark)[![Downloads](https://static.pepy.tech/personalized-badge/geospark?period=total&units=international_system&left_color=black&right_color=brightgreen&left_text=total%20downloads)](https://pepy.tech/project/geospark)                      |                                                                                                                                                 |                                                                                                                                                                                                                                                                                                             |                                                                                                                                |
 
 * [Join the community](#join-the-community)
 * [What is Apache Sedona?](#what-is-apache-sedona)
@@ -22,7 +22,7 @@
 
 Follow Sedona on Twitter for fresh news: [Sedona@Twitter](https://twitter.com/ApacheSedona)
 
-Join the Sedona Discord community: [![](https://dcbadge.vercel.app/api/server/9A3k5dEBsY)](https://share.hsforms.com/1Ndql_ZigTdmLlVQc_d1o4gqga4q)
+Join the Sedona Discord community: [![Apache Sedona Community Discord Server](https://dcbadge.vercel.app/api/server/9A3k5dEBsY)](https://share.hsforms.com/1Ndql_ZigTdmLlVQc_d1o4gqga4q)
 
 Join the Sedona monthly community office hour: [Google Calendar](https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=NjI0cWgwcTZndnI1anAzYnFrNHY5Y2wyaTRfMjAyNDA0MDlUMTUwMDAwWiBjX2VmN2Q1NGY1MzA4YTRiN2YyNWFjMzNkMGY3ZWViNTRhM2E3ZjExNWI2ODlmYWY0ZDgyNDI1ZjNjYjVlZGU5MzVAZw&tmsrc=c_ef7d54f5308a4b7f25ac33d0f7eeb54a3a7f115b689faf4d82425f3cb5ede935%40group.calendar.google.com&scp=ALL), Tuesdays from 8 AM to 9 AM Pacific Time, every 4 weeks
 
@@ -36,7 +36,7 @@ Join the Sedona monthly community office hour: [Google Calendar](https://calenda
 
 Apache Sedona™ is a spatial computing engine that enables developers to easily process spatial data at any scale within modern cluster computing systems such as Apache Spark and Apache Flink. Sedona developers can express their spatial data processing tasks in Spatial SQL, Spatial Python or Spatial R. Internally, Sedona provides spatial data loading, indexing, partitioning, and query processing/optimization functionality that enable users to efficiently analyze spatial data at any scale.
 
-![](docs/image/sedona-ecosystem.png "Sedona Ecosystem")
+![Sedona Ecosystem](docs/image/sedona-ecosystem.png "Sedona Ecosystem")
 
 ### Features
 

common/src/main/java/org/apache/sedona/common/utils/GeomUtils.java

@@ -344,7 +344,7 @@ public static boolean geometryIsPolygonal(Geometry geometry) {
   }
 
   /**
-   * Checks if the geoemetry pair - <code>left</code> and <code>right</code> - should be handled be
+   * Checks if the geometry pair - <code>left</code> and <code>right</code> - should be handled be
    * the current partition - <code>extent</code>.
    *
    * @param left

common/src/main/java/org/apache/sedona/common/utils/H3Utils.java

@@ -79,7 +79,7 @@ public H3UtilException(String message, Throwable cause) {
     cellSizeMap.put(15, 0.0004567448929842399);
   }
 
-  public static LatLng coordindateToLatLng(Coordinate coordinate) {
+  public static LatLng coordinateToLatLng(Coordinate coordinate) {
     return new LatLng(coordinate.getY(), coordinate.getX());
   }
 
@@ -96,13 +96,13 @@ public static List<Long> polygonToCells(Polygon polygon, int level, boolean full
     try {
       List<LatLng> shell =
           Arrays.stream(polygon.getExteriorRing().getCoordinates())
-              .map(H3Utils::coordindateToLatLng)
+              .map(H3Utils::coordinateToLatLng)
               .collect(Collectors.toList());
       List<List<LatLng>> holes = new ArrayList<>();
       for (int i = 0; i < polygon.getNumInteriorRing(); i++) {
         holes.add(
             Arrays.stream(polygon.getInteriorRingN(i).getCoordinates())
-                .map(H3Utils::coordindateToLatLng)
+                .map(H3Utils::coordinateToLatLng)
                 .collect(Collectors.toList()));
       }
       // H3 polyfill only include hexagons with centroid within the polygon, we fix by generating

docs/api/snowflake/vector-data/Function.md

@@ -2810,7 +2810,7 @@ Format: `ST_Snap(input: Geometry, reference: Geometry, tolerance: double)`
 
 Input geometry:
 
-![](../../../image/st_snap/st-snap-base-example.png "ST_Snap Base example")
+![ST_Snap Base example](../../../image/st_snap/st-snap-base-example.png "ST_Snap Base example")
 
 SQL Example:
 
@@ -2824,7 +2824,7 @@ SELECT
 
 Output:
 
-![](../../../image/st_snap/st-snap-applied.png "ST_Snap applied example")
+![ST_Snap applied example](../../../image/st_snap/st-snap-applied.png "ST_Snap applied example")
 
 ```
 POLYGON ((236877.58 -6.61, 236878.29 -8.35, 236879.98 -8.33, 236879.72 -7.63, 236880.69 -6.81, 236877.58 -6.61), (236878.45 -7.01, 236878.43 -7.52, 236879.29 -7.5, 236878.63 -7.22, 236878.76 -6.89, 236878.45 -7.01))

docs/api/sql/Optimizer.md

@@ -130,7 +130,7 @@ WHERE ST_DistanceSpheroid(pointdf1.pointshape1,pointdf2.pointshape2) <= 2
 ```
 
 !!!warning
-	If you use `ST_DistanceSpheroid ` or `ST_DistanceSphere` as the predicate, the unit of the distance is meter. Currently, distance join with geodesic distance calculators work best for point data. For non-point data, it only considers their centroids.
+	If you use `ST_DistanceSpheroid` or `ST_DistanceSphere` as the predicate, the unit of the distance is meter. Currently, distance join with geodesic distance calculators work best for point data. For non-point data, it only considers their centroids.
 
 ## Broadcast index join
 
@@ -340,8 +340,8 @@ answer queries such as `SELECT * FROM geoparquet_dataset WHERE ST_Intersects(geo
 
 We can compare the metrics of querying the GeoParquet dataset with or without the spatial predicate and observe that querying with spatial predicate results in fewer number of rows scanned.
 
-| Without spatial predicate | With spatial predicate |
-| ----------- | ----------- |
-| ![](../../image/scan-parquet-without-spatial-pred.png) | ![](../../image/scan-parquet-with-spatial-pred.png) |
+| Without spatial predicate                                                                       | With spatial predicate |
+|-------------------------------------------------------------------------------------------------| ----------- |
+| ![Scan geoparquet without spatial predicate](../../image/scan-parquet-without-spatial-pred.png) | ![Scan geoparquet with spatial predicate](../../image/scan-parquet-with-spatial-pred.png) |
 
 Spatial predicate push-down to GeoParquet is enabled by default. Users can manually disable it by setting the Spark configuration `spark.sedona.geoparquet.spatialFilterPushDown` to `false`.

docs/community/contact.md

@@ -19,7 +19,7 @@ Everyone is welcome to join our community events. We have a community office hou
 
 ## Discord Server
 
-[![](https://dcbadge.vercel.app/api/server/9A3k5dEBsY)](https://share.hsforms.com/1Ndql_ZigTdmLlVQc_d1o4gqga4q)
+[![Apache Sedona Community Discord Server](https://dcbadge.vercel.app/api/server/9A3k5dEBsY)](https://share.hsforms.com/1Ndql_ZigTdmLlVQc_d1o4gqga4q)
 
 ## Mailing list