-
Notifications
You must be signed in to change notification settings - Fork 693
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SEDONA-323] Add keplergl wrapper #898
Merged
jiayuasu
merged 25 commits into
apache:master
from
iGN5117:develop_Nilesh_1.5.0_NotebookVisualization
Jul 18, 2023
Merged
Changes from 18 commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
16f39f5
Update apache sedona version to 1.4.1
iGN5117 c132894
Refactored notebook imports and added unified SedonaContext entry points
iGN5117 ff9238c
Replaced geopandas plot with KeplerGL visualization.
iGN5117 38ae4bd
Merge branch 'master' into develop_Nilesh_1.5.0_NotebookVisualization
iGN5117 40135b3
Add keplerGL to pipfile
iGN5117 c7a62a2
Merge branch 'develop_Nilesh_1.5.0_NotebookVisualization' of https://…
iGN5117 6629120
Merge branch 'master' into develop_Nilesh_1.5.0_NotebookVisualization
iGN5117 3095099
try adding dependencies in pipfile
iGN5117 10adc55
Merge branch 'develop_Nilesh_1.5.0_NotebookVisualization' of https://…
iGN5117 088aaf4
Add keplergl import
iGN5117 8e22147
Add env.yml for binder (copied from leafmap)
iGN5117 d9d3cbb
Force version 3.6.4 of jupyter lab in pipfile
iGN5117 196ac82
revert adding environment.yml
iGN5117 0f0015c
add sedona.maps with SedonaKepler, a wrapper for KeplerGl visualizati…
iGN5117 92578dc
Merge branch 'sedona-master' into develop_Nilesh_1.5.0_NotebookVisual…
iGN5117 bb46de5
Refactor SedonaKepler wrapper to take SedonaDataFrames as data input
iGN5117 349c855
moved sedonakepler tests to a separate folder
iGN5117 a566ee0
Add apache license header to newly created py files
iGN5117 f7964ba
refactor private method signature to be in line with naming convention
iGN5117 5786f2a
Added comment explaining usage of _repr_html()
iGN5117 4ce9f22
Update documentation
iGN5117 084e0dd
fix typo
iGN5117 b4295e2
Add copies of datasets to get around keplergl bug
iGN5117 2e822bf
Refactor SedonaKepler constructor to add support for pandas with pyth…
iGN5117 bea9973
Missed pushing gif
iGN5117 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
346 changes: 327 additions & 19 deletions
346
binder/ApacheSedonaSQL_SpatialJoin_AirportsPerCountry.ipynb
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jiayuasu marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,75 @@ | ||
# Licensed to the Apache Software Foundation (ASF) under one | ||
# or more contributor license agreements. See the NOTICE file | ||
# distributed with this work for additional information | ||
# regarding copyright ownership. The ASF licenses this file | ||
# to you under the Apache License, Version 2.0 (the | ||
# "License"); you may not use this file except in compliance | ||
# with the License. You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, | ||
# software distributed under the License is distributed on an | ||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
# KIND, either express or implied. See the License for the | ||
# specific language governing permissions and limitations | ||
# under the License. | ||
|
||
from keplergl import KeplerGl | ||
import geopandas as gpd | ||
Kontinuation marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
|
||
class SedonaKepler: | ||
|
||
@classmethod | ||
def create_map(cls, df=None, name="unnamed", geometry_col="geometry", config=None): | ||
""" | ||
Creates a map visualization using kepler, optionally taking a sedona dataFrame as data input | ||
:param df: [Optional] SedonaDataFrame to plot on the map | ||
:param name: [Optional] Name to be associated with the given dataframe, if a df is passed with no name, a default name of 'unnamed' is set for it. | ||
:param geometry_col: [Optional] Custom name of geometry column in the sedona data frame, | ||
if no name is provided, it is assumed that the column has the default name 'geometry'. | ||
:param config: [Optional] A map config to be applied to the rendered map | ||
:return: A map object | ||
""" | ||
if df is not None: | ||
geoDf = SedonaKepler._convertToGdf(df, geometry_col) | ||
dataDict = {name: geoDf} | ||
map = KeplerGl(data=dataDict) | ||
else: | ||
map = KeplerGl() | ||
|
||
if config is not None: | ||
map.config = config | ||
|
||
return map | ||
|
||
@classmethod | ||
def add_df(cls, map, df, name="unnamed", geometry_col="geometry"): | ||
""" | ||
Adds a SedonaDataFrame to a given map object. | ||
:param map: Map object to add SedonaDataFrame to | ||
:param df: SedonaDataFrame to add | ||
:param name: [Optional] Name to assign to the dataframe, default name assigned is 'unnamed' | ||
:param geometry_col: [Optional] Custom name of geometry_column if any, if no name is provided, a default name of 'geometry' is assumed. | ||
:return: Does not return anything, adds df directly to the given map object | ||
""" | ||
geoDf = SedonaKepler._convertToGdf(df, geometry_col) | ||
map.add_data(geoDf, name=name) | ||
|
||
|
||
@classmethod | ||
def _convertToGdf(cls, df, geometry_col="geometry"): | ||
""" | ||
Converts a SedonaDataFrame to a GeoPandasDataFrame and also renames geometry column to a standard name of 'geometry' | ||
:param df: SedonaDataFrame to convert | ||
:param geometry_col: [Optional] | ||
:return: | ||
""" | ||
pandasDf = df.toPandas() | ||
geoDf = gpd.GeoDataFrame(pandasDf, geometry=geometry_col) | ||
if geometry_col != "geometry": | ||
geoDf = geoDf.rename(columns={geometry_col: "geometry"}) | ||
return geoDf | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# Licensed to the Apache Software Foundation (ASF) under one | ||
# or more contributor license agreements. See the NOTICE file | ||
# distributed with this work for additional information | ||
# regarding copyright ownership. The ASF licenses this file | ||
# to you under the Apache License, Version 2.0 (the | ||
# "License"); you may not use this file except in compliance | ||
# with the License. You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, | ||
# software distributed under the License is distributed on an | ||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
# KIND, either express or implied. See the License for the | ||
# specific language governing permissions and limitations | ||
# under the License. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# Licensed to the Apache Software Foundation (ASF) under one | ||
# or more contributor license agreements. See the NOTICE file | ||
# distributed with this work for additional information | ||
# regarding copyright ownership. The ASF licenses this file | ||
# to you under the Apache License, Version 2.0 (the | ||
# "License"); you may not use this file except in compliance | ||
# with the License. You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, | ||
# software distributed under the License is distributed on an | ||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
# KIND, either express or implied. See the License for the | ||
# specific language governing permissions and limitations | ||
# under the License. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
# Licensed to the Apache Software Foundation (ASF) under one | ||
# or more contributor license agreements. See the NOTICE file | ||
# distributed with this work for additional information | ||
# regarding copyright ownership. The ASF licenses this file | ||
# to you under the Apache License, Version 2.0 (the | ||
# "License"); you may not use this file except in compliance | ||
# with the License. You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, | ||
# software distributed under the License is distributed on an | ||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
# KIND, either express or implied. See the License for the | ||
# specific language governing permissions and limitations | ||
# under the License. | ||
|
||
from keplergl import KeplerGl | ||
from sedona.maps.SedonaKepler import SedonaKepler | ||
from tests.test_base import TestBase | ||
|
||
|
||
class TestVisualization(TestBase): | ||
Kontinuation marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
def test_map_creation(self): | ||
sedona_kepler_map = SedonaKepler.create_map() | ||
kepler_map = KeplerGl() | ||
assert sedona_kepler_map.config == kepler_map.config |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@iGN5117 This doc references to this image but this image was not committed together with this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry about that, pushed the gif