Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

22 onglet data ameliorations #29

Merged
merged 28 commits into from
Jul 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
fb186a5
[tg] - valeurs par défaut des filtres et optimisation du layout des f…
tgazagnes May 22, 2024
2ad8ba5
[tg] - convert liters to m3
tgazagnes May 22, 2024
47049af
[tg] - wording sous onglet matériaux
tgazagnes May 22, 2024
1c9d037
[tg] - classement par ordre alphabétique dans les filtres
tgazagnes May 22, 2024
b941905
[tg] - wording sous-onglet top déchets
tgazagnes May 22, 2024
c19f74b
[tg] - top déchets : passage rayon en log2
tgazagnes May 23, 2024
55e4894
[tg] - chart labels templates
tgazagnes May 23, 2024
5c17732
[tg] - amélioration format nombres utilisant locale.set_locale
tgazagnes May 24, 2024
042e133
[tg] - bug corrigé
tgazagnes May 24, 2024
f224645
[tg] - note méthodo et exclusion des 100%Autres
tgazagnes May 24, 2024
e1c7a80
[tg} - infobulles dans les graphiques en langage naturel
tgazagnes May 27, 2024
dc7a302
[tg] - onglet matériaux : tableau avec nb collectes par milieu
tgazagnes May 27, 2024
34b0996
[tg] - onglet matériaux - améliorations table nb déchets
tgazagnes May 27, 2024
9f1b0e4
[tg] - séparateurs décimaux et milliers dans les graphiques
tgazagnes May 29, 2024
0835d03
[tg] - sous-onglet matériaux : filtrer sur milieux avec plus de 3 col…
tgazagnes May 29, 2024
9225aea
[tg] - ajout des filtres sur top10 déchets
tgazagnes May 29, 2024
3f24729
[tg] inversion graphs marques et REP
tgazagnes May 29, 2024
5334ee7
[tg] - sous-onglet 3 : filtre sur niveaux 4 (secteurs/REP) et 2à4 (ma…
tgazagnes May 29, 2024
4c048eb
[tg] - retrait des VIDES et INDETERMINES dans les metrics de l'onglet 3
tgazagnes May 29, 2024
f7dc00c
[tg] - ajout %déchets dans graph par secteurs
tgazagnes May 29, 2024
fb43f53
[tg] - amélioration layout 3 onglets
tgazagnes Jun 6, 2024
a9ff5f1
[tg] - optimisation du code des filtres
tgazagnes Jun 6, 2024
858bb7a
[tg] Améliorations suite point Merterre
tgazagnes Jun 6, 2024
0c42b3b
[tg] Ajout détail données utilisées + rayon carto top déchets
tgazagnes Jun 18, 2024
9ddfbed
[tg] remove line from poetry.lock to solve precommit fail
tgazagnes Jun 19, 2024
ba83666
[tg] update poetry.lock to solve precommit conflicts
tgazagnes Jun 20, 2024
8cc6880
[tg] black reformat to hotspots.py
tgazagnes Jun 20, 2024
e919f14
[tg] corrections suite retours Kyllian
tgazagnes Jun 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ ipython_config.py
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock
poetry.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
Expand Down
1,643 changes: 931 additions & 712 deletions dashboards/app/pages/data.py

Large diffs are not rendered by default.

100 changes: 61 additions & 39 deletions dashboards/app/pages/hotspots.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pourquoi l'onglet hotspot est modifié, si l'on modifie data ?

Copy link
Collaborator Author

@tgazagnes tgazagnes Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

C'est precommit qui a fait passer du reformatage (black je pense vu les modifs), je pense qu'ils ne l'ont pas utilisé dans les derniers commit du staging. Je les laisserai écraser les modifs avec leur dernière version pré-commitée :). Aucune modif volontaire de ma part

Original file line number Diff line number Diff line change
Expand Up @@ -432,25 +432,30 @@ def calculate_and_display_metrics(data, indicator_col1, indicator_col2, indicato
def couleur_milieu(type):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prévoir du type hinting

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

à voir avec l'onglet hotspots

return couleur.get(type, "white") # Returns 'white' if the type is not found


def update_lieu_options(selected_milieu):
if selected_milieu and selected_milieu != "Sélectionnez un milieu...":
filtered_data = data_zds[data_zds['TYPE_MILIEU'] == selected_milieu]
return ["Sélectionnez un lieu..."] + list(filtered_data['TYPE_LIEU2'].dropna().unique())
filtered_data = data_zds[data_zds["TYPE_MILIEU"] == selected_milieu]
return ["Sélectionnez un lieu..."] + list(
filtered_data["TYPE_LIEU2"].dropna().unique()
)
return ["Sélectionnez un lieu..."]


@st.cache_data
def process_data(data_zds):
# Filtering data to ensure surface area is not zero
data_zds = data_zds[data_zds['SURFACE'] > 0]
data_zds = data_zds[data_zds["SURFACE"] > 0]
# Calculating density and filtering out anomalous values
data_zds['DENSITE'] = data_zds['VOLUME_TOTAL'] / data_zds['SURFACE']
data_zds = data_zds[data_zds['DENSITE'] < 20]
data_zds["DENSITE"] = data_zds["VOLUME_TOTAL"] / data_zds["SURFACE"]
data_zds = data_zds[data_zds["DENSITE"] < 20]
# Rounding values for better display
data_zds['DENSITE'] = data_zds['DENSITE'].round(4)
data_zds['SURFACE_ROND'] = data_zds['SURFACE'].round(2)
data_zds["DENSITE"] = data_zds["DENSITE"].round(4)
data_zds["SURFACE_ROND"] = data_zds["SURFACE"].round(2)
return data_zds

#Zoom from admin level

# Zoom from admin level
if NIVEAU_ADMIN == "Commune":
zoom_admin = 12
elif NIVEAU_ADMIN == "EPCI":
Expand All @@ -469,14 +474,16 @@ def plot_density_map(data_zds: pd.DataFrame, filtered_data: pd.DataFrame) -> fol

else:
# Use processed data
processed_data = process_data(filtered_data if not filtered_data.empty else data_zds)
processed_data = process_data(
filtered_data if not filtered_data.empty else data_zds
)

m = folium.Map(
location=[
processed_data['LIEU_COORD_GPS_Y'].mean(),
processed_data['LIEU_COORD_GPS_X'].mean()
processed_data["LIEU_COORD_GPS_Y"].mean(),
processed_data["LIEU_COORD_GPS_X"].mean(),
],
zoom_start=zoom_admin
zoom_start=zoom_admin,
)

# Loop over each row in the DataFrame to place markers
Expand All @@ -491,26 +498,25 @@ def plot_density_map(data_zds: pd.DataFrame, filtered_data: pd.DataFrame) -> fol
</div>
"""
lgd_txt = '<span style="color: {col};">{txt}</span>'
color = couleur_milieu(row['TYPE_MILIEU'])
color = couleur_milieu(row["TYPE_MILIEU"])
folium.CircleMarker(
fg = folium.FeatureGroup(name= lgd_txt.format( txt= ['TYPE_MILIEU'], col= color)),
location=[row['LIEU_COORD_GPS_Y'], row['LIEU_COORD_GPS_X']],
radius=np.log(row['DENSITE'] + 1)*15,
fg=folium.FeatureGroup(
name=lgd_txt.format(txt=["TYPE_MILIEU"], col=color)
),
location=[row["LIEU_COORD_GPS_Y"], row["LIEU_COORD_GPS_X"]],
radius=np.log(row["DENSITE"] + 1) * 15,
popup=folium.Popup(popup_html, max_width=300),
color=color,
fill=True,

).add_to(m)

folium_static(m)

return m


# Function for 'milieu' density table
def density_table_milieu(
data_zds: pd.DataFrame,
filtered_data: pd.DataFrame
):
def density_table_milieu(data_zds: pd.DataFrame, filtered_data: pd.DataFrame):

if data_zds.empty:
st.write("Aucune donnée disponible pour la région sélectionnée.")
Expand Down Expand Up @@ -554,10 +560,7 @@ def density_table_milieu(
)


def density_table_lieu(
data_zds: pd.DataFrame,
filtered_data: pd.DataFrame
):
def density_table_lieu(data_zds: pd.DataFrame, filtered_data: pd.DataFrame):

if data_zds.empty:
st.write("Aucune donnée disponible pour la région sélectionnée.")
Expand Down Expand Up @@ -766,55 +769,74 @@ def create_contributors_table(data_zds: pd.DataFrame, multi_filter_dict: dict) -
# Add a default "Select a milieu..." option
selected_milieu = st.selectbox(
"Sélectionnez un milieu:",
["Sélectionnez un milieu..."] + list(pd.unique(data_zds_correct['TYPE_MILIEU']))
["Sélectionnez un milieu..."]
+ list(pd.unique(data_zds_correct["TYPE_MILIEU"])),
)
with right_column:
# Update lieu options based on selected milieu
lieu_options = update_lieu_options(selected_milieu)
selected_lieu = st.selectbox("Sélectionnez un lieu:", lieu_options)


# Place the map centrally by using a wider column for the map and narrower ones on the sides
col1, map_col, col3 = st.columns([4, 10, 1]) # Adjust column ratios as needed

with map_col:
st.markdown("### Carte des Densités")
if selected_milieu != "Sélectionnez un milieu..." and selected_lieu != "Sélectionnez un lieu...":
filtered_data = data_zds_correct[(data_zds_correct['TYPE_MILIEU'] == selected_milieu) & (data_zds_correct['TYPE_LIEU2'] == selected_lieu)]
if (
selected_milieu != "Sélectionnez un milieu..."
and selected_lieu != "Sélectionnez un lieu..."
):
filtered_data = data_zds_correct[
(data_zds_correct["TYPE_MILIEU"] == selected_milieu)
& (data_zds_correct["TYPE_LIEU2"] == selected_lieu)
]
plot_density_map(data_zds_correct, filtered_data)
else:
plot_density_map(data_zds_correct, data_zds_correct) # Show all data by default

plot_density_map(
data_zds_correct, data_zds_correct
) # Show all data by default

col1, col2, col3 = st.columns([3, 3, 2])

with col1:
st.markdown("#### Tableau des Densités par Milieu")
if selected_milieu != "Sélectionnez un milieu..." and selected_lieu != "Sélectionnez un lieu...":
filtered_data = data_zds_correct[(data_zds_correct['TYPE_MILIEU'] == selected_milieu) & (data_zds_correct['TYPE_LIEU2'] == selected_lieu)]
if (
selected_milieu != "Sélectionnez un milieu..."
and selected_lieu != "Sélectionnez un lieu..."
):
filtered_data = data_zds_correct[
(data_zds_correct["TYPE_MILIEU"] == selected_milieu)
& (data_zds_correct["TYPE_LIEU2"] == selected_lieu)
]
density_table_milieu(data_zds_correct, filtered_data)
else:
density_table_milieu(data_zds_correct, data_zds_correct)

with col2:
st.markdown("#### Tableau des Densités par Lieu")
if selected_milieu != "Sélectionnez un milieu..." and selected_lieu != "Sélectionnez un lieu...":
filtered_data = data_zds_correct[(data_zds_correct['TYPE_MILIEU'] == selected_milieu) & (data_zds_correct['TYPE_LIEU2'] == selected_lieu)]
if (
selected_milieu != "Sélectionnez un milieu..."
and selected_lieu != "Sélectionnez un lieu..."
):
filtered_data = data_zds_correct[
(data_zds_correct["TYPE_MILIEU"] == selected_milieu)
& (data_zds_correct["TYPE_LIEU2"] == selected_lieu)
]
density_table_lieu(data_zds_correct, filtered_data)
else:
density_table_lieu(data_zds_correct, data_zds_correct)

with col3:
with st.expander("###### Notice ℹ️", expanded=True):
st.write(
"""
with st.expander("###### Notice ℹ️", expanded=True):
st.write(
"""
**Milieu** désigne de grands types d'environnements comme le Littoral,
les Cours d'eau ou la Montagne.\n
Chaque Milieu est ensuite divisé en
**Lieux** plus spécifiques. Par exemple, sous le Milieu Littoral,
on trouve des Lieux comme les Plages, les Roches, les Digues, ou les Parkings.
"""
)
)

with tab2:
# Use the selected filters
Expand Down
1 change: 1 addition & 0 deletions dashboards/app/pages/ongletdata_colormap_materiaux.json
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note pour plus tard : Si l'on a d'autres export de colormap ou autre, autant tout centraliser dans un json ?

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"Textile": "#C384B1", "Papier": "#CAA674", "Metal": "#A0A0A0", "Verre": "#3DCE89", "Autre": "#F3B900", "Plastique": "#48BEF0", "Caoutchouc": "#364E74", "Bois": "#673C11", "Papier/Carton": "#CAA674", "M\u00e9tal": "#A0A0A0", "Verre/C\u00e9ramique": "#3DCE89"}
1 change: 1 addition & 0 deletions dashboards/app/pages/ongletdata_colormap_secteurs.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"AGRICULTURE": "#156644", "ALIMENTATION": "#F7D156", "AMEUBLEMENT, D\u00c9CORATION ET \u00c9QUIPEMENT DE LA MAISON": "#F79D65", "AQUACULTURE": "#0067C2", "B\u00c2TIMENT, TRAVAUX ET MAT\u00c9RIAUX DE CONSTRUCTION": "#FF9900", "CHASSE ET ARMEMENT": "#23A76F", "COSM\u00c9TIQUES, HYGI\u00c8NE ET SOINS PERSONNELS": "#BF726B", "D\u00c9TERGENTS ET PRODUITS D'ENTRETIENS": "#506266", "EMBALLAGE INDUSTRIEL ET COLIS": "#754B30", "GRAPHIQUE ET PAPETERIE ET FOURNITURES DE BUREAU": "#EFEFEF", "IND\u00c9TERMIN\u00c9": "#967EA1", "INFORMATIQUE ET HIGHTECH": "#E351F7", "JOUETS ET LOISIR": "#A64D79", "MAT\u00c9RIEL \u00c9LECTRIQUE ET \u00c9LECTROM\u00c9NAGER": "#AE05C3", "M\u00c9TALLURGIE": "#EC4773", "P\u00caCHE": "#003463", "PETROCHIMIE": "#0D0D0D", "PHARMACEUTIQUE/PARAM\u00c9DICAL": "#61BF5E", "PLASTURGIE": "#05A2AD", "TABAC": "#E9003F", "TEXTILE ET HABILLEMENT": "#FA9EE5", "TRAITEMENT DES EAUX": "#4AA6F7", "TRANSPORT / AUTOMOBILE": "#6C2775", "VAISSELLE \u00c0 USAGE UNIQUE": "#732D3A", "AUTRES SECTEURS": "#D9C190"}
1 change: 0 additions & 1 deletion dashboards/app/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ streamlit==1.32.2
openpyxl==3.1.2
streamlit-folium==0.19.1
plotly==5.19.0
streamlit-dynamic-filters==0.1.6
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pas d'impact sur les autres onglets?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope on l'avait testé sur data et remplacé ensuite. aucun autre onglet ne l'utilise

streamlit-authenticator==0.3.2
st-pages==0.4.5
babel==2.11.0
Loading
Loading