Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Site multiprotocole + CRUVED #264

Open
wants to merge 241 commits into
base: develop
Choose a base branch
from
Open

Site multiprotocole + CRUVED #264

wants to merge 241 commits into from

Conversation

amandine-sahl
Copy link
Contributor

@amandine-sahl amandine-sahl force-pushed the dev-suivi-eolien branch 2 times, most recently from bd4a8c0 to 4b2c4d5 Compare December 8, 2023 16:48
Copy link

codecov bot commented Dec 8, 2023

Codecov Report

Attention: Patch coverage is 78.68339% with 272 lines in your changes missing coverage. Please review.

Please upload report for BASE (develop@b1e625f). Learn more about missing BASE report.

Current head b83975c differs from pull request most recent head 3c4274d

Please upload reports for the commit 3c4274d to get more accurate results.

Files Patch % Lines
backend/gn_module_monitoring/monitoring/queries.py 70.52% 56 Missing ⚠️
backend/gn_module_monitoring/monitoring/models.py 62.90% 46 Missing ⚠️
backend/gn_module_monitoring/routes/monitoring.py 51.28% 38 Missing ⚠️
backend/gn_module_monitoring/utils/routes.py 79.82% 23 Missing ⚠️
...kend/gn_module_monitoring/monitoring/serializer.py 72.60% 20 Missing ⚠️
...ackend/gn_module_monitoring/routes/sites_groups.py 77.64% 19 Missing ⚠️
...nd/gn_module_monitoring/monitoring/repositories.py 34.78% 15 Missing ⚠️
backend/gn_module_monitoring/routes/site.py 89.55% 14 Missing ⚠️
backend/gn_module_monitoring/monitoring/admin.py 76.00% 12 Missing ⚠️
backend/gn_module_monitoring/monitoring/objects.py 50.00% 6 Missing ⚠️
... and 7 more
Additional details and impacted files
@@            Coverage Diff             @@
##             develop     #264   +/-   ##
==========================================
  Coverage           ?   75.59%           
==========================================
  Files              ?       30           
  Lines              ?     2270           
  Branches           ?        0           
==========================================
  Hits               ?     1716           
  Misses             ?      554           
  Partials           ?        0           
Flag Coverage Δ
pytest 75.59% <78.68%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@MathRdt
Copy link

MathRdt commented Dec 8, 2023

Bonjour,
je me suis permis de tester cette nouvelle feature de gestion de la portée pour les utilisateurs, merci pour le travail et la refonte de la plupart des routes de monitoring !

Cette gestion fonctionne bien quand on appelle directement l'objet en question, mais semble passer au travers des mailles du filet via le "depth" de l'objet parent qui permet tout de même de visualiser des objets sur lesquels je n'ai pas de droits.

Exemple:
J'ai un rôle qui me permet de visualiser tous les groupes de sites, mais uniquement "mes sites" dans mon sous-module compte_ponte.
Lorsque j'appelle la route /geonature/api/monitorings/object/compte_ponte/sites_group/38?depth=1
je vais quand même récupérer l'intégralité des sites dans le "children" de mon body, même ceux qui ne "m'appartiennent" pas.
Par contre, je ne peux effectivement pas appeler directement ces sites via l'API et je prends une 403.

Est ce un comportement voulu ?

De mon côté, je pense que c'est une petite faille de sécurité pour GET des objets auxquels on est pas censés avoir accès.
Vu la façon dont sont construits les modèles, je me doute que c'est une correction qui risque de ne pas être si simple que ça à apporter...

Maxime Vergez and others added 23 commits July 29, 2024 10:54
commit 70b0139
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 15:22:56 2022 +0100

    style: apply black

commit 2badedf
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 15:11:41 2022 +0100

    chore(db): removed unused import

commit 57d9b52
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 14:47:48 2022 +0100

    feat(db): improved & finished downgrade func

commit 211780f
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 09:32:12 2022 +0100

    feat(db): wip: add upgrade and began downgrade

    Downgrade migration need a little bit more work due to the use of
    sql alchemy and not plain text sql

commit 0b5a3b8
Author: Maxime Vergez <[email protected]>
Date:   Mon Dec 5 11:40:32 2022 +0100

    feat: packaged module
Squashed commit of the following:

commit 2bdc22a
Author: Maxime Vergez <[email protected]>
Date:   Fri Dec 9 14:59:31 2022 +0100

    chore: changed json_schema into config

commit 4279389
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 18:37:41 2022 +0100

    feat(db): add foreign key from site to cat

    From t_base_sites to bib_categorie_site

commit 5cb5dfa
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 16:47:27 2022 +0100

    feat(api): add admin interface

commit 559a0ec
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 16:46:27 2022 +0100

    feat(db): add bib_categorie_site table

    Add migration and model
…t description #2 (#15)

* feat: [6.2] Page d'accueil modul monitoring layout et config title et description #2

* chore(config): applied black & removed imports
* feat(api): wip began add site routes + tests

With site categories
Also add tests

* feat(api): add more routes

* test(api): add tests and fixtures

* style(api): applied black

* feat(db): add migration to remove id_module

Column in t_sites_groups

* refactor(api): move utils for routes from sites

* feat(api): wip: add sites groups route

* test(api): wip: begin adding fixture site_groups

* fix: remove id_module in all models

* chore: rename route for better consistency

* tests: moved site_groups in tests and add tests

* chore(api): applied black

* refactor(api): add filter params function

And refact routes to use it
* feat(api): add association table with alembic

Add model in backend and alembic migration

Reviewed-by: andriac
[Refs ticket]: #3

* test: WIP add test to see new relationship

Adding test to see if categories are showing up when we call module

Reviewed-by: andriacap
[Refs ticket]: #3

* feat: add type site - categorie relation

WIP - add selectfield to get type site in admin module

Reviewed-by: andriac
[Refs ticket]: #3

* feat(api): Flask admin and routes categories

Clean code for change label list and form selectfield for
the BibCategorieView in Flask Admin

Add utils routes to get all subtable relationship in order to
get back the label type site

Review-by: andriac
[Refs ticket]: #3

* refactor: remove paginate_nested

For depth in as_dict()

* test: fix tests due to as_dict depth

* style: applied black and isort

* chore: remove unused import

Co-authored-by: Andria Capai <[email protected]>
* tests: make tests and fixtures work for modules

By changing the way a monitoring module is created in the fixture
Add two tests to check the relationship

* feat(api): add categories in edit module

* style: applied black
Need to set the current working directory of pytest the directory
where the gn_monitoring_module repo is (so with the frontend etc.)
clean: Merge la PR move test_route 
Co-authored-by: Andria Capai <[email protected]>
* feat(api): remove id_module from sites_complements

Co-authored-by: andriacap <[email protected]>

* feat(api): create schema for sites_groups

* test: add sites_group to site fixture

* test: wip add test for sites_groups schemas

* chore(api): remove depth parameter from paginate

* test: updated to work with sites_group schema

* feat: categorie site with marshmallow

Test and marshmallow create/refactor to
adapt for bibcategorie site paginate

WIP : Adapt load_only site_type in test to "assert"
same object when initiate BibCategorieSite

[Refs ticket]: #3

* feat(api): route /sites/categories/id with schema

Changing the route to return a dump Marshmallow schema
BibCategorieSitesSchema

Reviewed-by: andriac
[Refs ticket]: #3

* test(api): routes get categoires label

Change the "as_dict" by schema.dump in order
to use the Marshmallow schema created

Reviewed-by: andriac
[Refs ticket]: #3

* feat(api): Sites: cols to geoserializable + schema

* style(api): applied black

* test(api): add test for Site Schema

* style(api): applied black to test_site

* refactor(api): instantiate schema once

Instead of for each all() iteration

* chore(api): remove useless comments

* chore(api): remove useless comments and imports

Co-authored-by: andriacap <[email protected]>
Co-authored-by: Andria Capai <[email protected]>
To replace it with a bib nomenclature type: TYPE_SITE
* fix(api): add int conversion for limit/offset

* test(api): add test for get_limit_offset

* fix(api): max_per_page => per_page & test

* test(api): update test with changes on schema

Since marshmallow schemas, the json returned by pagination has
changed for items and not sites
* feat(api): add Query Class to sites, grps & cats

* feat(api): add sort and fix _get_model

Via _get_entity

* test(api): test sort query

* feat(api): add api sort/sort_dir params

To be able to sort through REST Api

* fix(api): check if integer to avoid using ilike

In filter_by_params

* test(api): add test to check filter integer
* Merged feat/package-module-alembic

commit 70b0139
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 15:22:56 2022 +0100

    style: apply black

commit 2badedf
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 15:11:41 2022 +0100

    chore(db): removed unused import

commit 57d9b52
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 14:47:48 2022 +0100

    feat(db): improved & finished downgrade func

commit 211780f
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 09:32:12 2022 +0100

    feat(db): wip: add upgrade and began downgrade

    Downgrade migration need a little bit more work due to the use of
    sql alchemy and not plain text sql

commit 0b5a3b8
Author: Maxime Vergez <[email protected]>
Date:   Mon Dec 5 11:40:32 2022 +0100

    feat: packaged module

* Feat/edit categories module (#19)

* tests: make tests and fixtures work for modules

By changing the way a monitoring module is created in the fixture
Add two tests to check the relationship

* feat(api): add categories in edit module

* style: applied black

* fix(config): changed categories into items

Following the change with marshmallow schemas
* feat(db): upgrade down_revision following rebase

Since rebase with develop: changed the down_revision number

* fix(db): fix bind params enabling downgrade

Beforehand the downgrade was not possible...

* refactor(db): removed cor_site_type_category

* refactor(db): changed category into type in cor

* refactor(db): create cor_type_site

* fix(db): renamed column

* refactor(api): update models to fit migrations

* fix(db):change bib_categorie_site to bib_type_site

Adding :
cor_site_module
cor_site_type
revision alembic to create function and trigger in order to add
bib_type_site but only with nomenclature 'TYPE_SITE'
upgrade and downgrade works

[Refs ticket]: #3
Reviewed-by: andriac

* fix(api): updated models from migrations

* fix(api): wip: fix admin following migrations

* fix(api): update routes and tests

To match migration changes

* feat: flask admin bib_type_site

Change bib_categories to bib_type_site into flask admin
Adding filtering in list label_fr of type_site to secure the unique
constraint

Reviewed-by: andriac
[Refs ticket]: #3

* fix(api): updated schema to match models

* fix(api): module edition

* style(api): uniformize type_site

* style(api): change relationship name for type_site

* feat(api): validator admin

* fix(api): make unique BibTypeSite in admin

* test(api): fix test when existing nomenclatures

In database

Co-authored-by: Andria Capai <[email protected]>
* Merged feat/package-module-alembic

commit 70b0139
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 15:22:56 2022 +0100

    style: apply black

commit 2badedf
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 15:11:41 2022 +0100

    chore(db): removed unused import

commit 57d9b52
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 14:47:48 2022 +0100

    feat(db): improved & finished downgrade func

commit 211780f
Author: Maxime Vergez <[email protected]>
Date:   Tue Dec 6 09:32:12 2022 +0100

    feat(db): wip: add upgrade and began downgrade

    Downgrade migration need a little bit more work due to the use of
    sql alchemy and not plain text sql

commit 0b5a3b8
Author: Maxime Vergez <[email protected]>
Date:   Mon Dec 5 11:40:32 2022 +0100

    feat: packaged module

* perf(api): improved loading of modules

When there are loads of datasets, the relationship takes a lot of
time to load, and is useless for the route that uses it.

* fix: due to wrong rebase

Fix: newline on data_utils.py
Fix: relationships are not joined anymore since there is no need of
them
Chore: update comment
* fix(api): page - 1 that returned the wrong page

* test(api): add test for paginate
* fix(db): change trigger to constraint (migrations)

Delete the trigger and create check constraint on id_nomenclature column

Reviewed-by: andriacap
[Refs ticket]: #3

* fix(db) : apply black on migration file

Apply black
[Refs_ticket]: #3
So that it will fallback to == most of the time and ilike when
just Unicode
Cannot use alembic to do this because need sqlalchemy >= 1.4.32
* refactor(api): change offset to page

* refactor(api): rename id_nomenclature

* fix(admin): add compare field to Unique

Since now, BibTypeSite and TNomenclature do not share the same
column anymore (id_nomenclature_type_site vs id_nomenclature)
amandine-sahl and others added 12 commits July 29, 2024 11:07
* Unify config

* Use cor_type_site data in config

* Use new config in frontend

* [DEV-SUIIVI-EOLIEN] refact: remove duplicate formValue (#328)

* refact: remove duplicate formValue

Use only formValue created in formService

Reviewed-by: andriacap

* style: lint frontend

Apply prettier

Reviewed-by: andriacap

* Remove customSpecConfig

* Dict update deeply

* Limitation de l'appel à get_config

* lint

* Add test module config

---------

Co-authored-by: andriacap <[email protected]>
* Remove customSpecConfig

* Limitation de l'appel à get_config

* Add test module config

* refact(wip): type site config with obj

Reviewed-by: andriacap

* refact(wip): init typeSite, generic and specific conf

Reviewed: andriaca

* refact(wip): make observable to init form componen

Reviewed-by: andriacap

* fix: change EMPTY to of(null)

Change return observable EMPTY by of(null)
in order to keep chained observable

Reviewed-by: andriacap

* refact(wip) : refact form component

Remove old methods
Use loop on dynamicForm

WIP: problem on initialize formControl (see
geonature/frontend/src/app/GN2CommonModule/form/dynamic-form-generator/dynamic-form-generator.component.ts)

Reviewed-by: andriacap

* refact: delete "dataComplement"

Remove dataComplement on form component
Remove dataComplement in backend on create_or_update
(called and used in pre_process) --> Check if
"TODO" code if is still necessary

Reviewed-by: andriacap

* refact:add extra dynamic form

On CREATION SITE
- Create objFormsDynamic
- Loop on these objFormsDynamic and display name for each group
- Update values for theses objFormsDynamic and link to objForm

TODO: Check if  EDITION SITE and other object type works

Reviewed-by: andriacap

* style: apply linter prettier

Reviewed-by: andriacap

* fix: unused code and fix condition

- Change condition to initFormDynamic
- Change condition to change typeSite on formValue

Reviewed-by: andriacap

* refact: changed way to subscribe to dynamicForm

- Remove place where variable called
- Centralize on subscribe of length of dynamicForm Array

Reviewed-by: andriacap

* fix: problem on init objFormDynamic on Edit object

- Need to filtered on types_site link to obj.properties['types_site']
- Add method to filtered (TODO: check if exist everywhere in utils ?)

Reviewed-by: andriacap

* fix/refact: change the way to initialize objForm

- Fix Problem on specific operator (the forkjoin didn't work)
- No more used of forkjoin and use defer in order to wait for specificValues$
is well completed before to subscribe and apply formValue

Reviewed-by: andriacap

* refact: remove unused code and use JSonData type

Reviewed-by: andriacap

* refact/fix: code bloc onInit and fix keep changes

- Fix: Problem on delete type site in dynamic form (keep changes and
re initialize values same as begining of edition)
- Refact(wip) : code on onInit to be more readable

Reviewed-by: andriacap

* refact(wip): more readable by separate concerns

Reviewed-by: andriacap

* refact(wip): move method to formService and utils

Reviewed-by: andriacap

* refact/doc: add docs and move dynamicForm$

- Move outside initialize typeSiteConfig to generate objFomDynamic
- Add doc for all methods added

Reviewed-by: andriacap

* fix: error for object without dynamicFormArray

Add check if dynamicFormArray exist in objForm

Reviewed-by: andriac

* refact: remove unused/useless code/commençant

Reviewed-by: andriacap

* style: lint frontend

Reviewed-by: andriacap

* Simplification code

* fix: remove return observable for objFormdefintion

- Remove return observable and remove pipe operator

Reviewed-by: andriac

* fix: check if list of types_site are integer

- FIXME : need to send only ids of types site
and not types site as string (fix in backend)

Reviewed-by: andriac

* fix: get all properties of additional data

Reviewed-by: andriacap

* fix: keep types sites prop from other modules

Backend :Need to add some code in backend to store extra
properties in order to find them in frontend

Frontend (form-component) : Get extra properties
stores in "additional_keys" in order to target them
when properties are filtered in post/patch method.

Reviewed-by: andriacp

* fix: use ids type site update when submit

Reviewed-by: andriacap

* Remove form-g and change route names

* Black

* refactor additionnal fields type site

* format

* fix update

* fix routing errors

* fix routing

* Use form component for site creation

* Rename component

* fix display properties in site manager

* Ne pas tenir compte du module lors de la récupération des sites parents

* Remove form-g + Routing

* Add child from datatable-g

* Prettier

* Navigation depuis le groupe de site

* Bloquer l'ajout de visite depuis generic site create

* Add parent path

* Popup

* Prettier

* Breadcrumb

* Correction pytest

* Router

* change permission order

* do not filter by module if generic module

* add missing grp_site object to monitoring module

* don't call group site group if no grp site

* fix routing

---------

Co-authored-by: Andria Capai <[email protected]>
Co-authored-by: TheoLechemia <[email protected]>
Co-authored-by: Théo Lechémia <[email protected]>
* Emission d'une notification lors de la suppression d'un élément d'une liste

* Typo

* Reset param forceReload
* add modules relationship on grp_site + refactor query

* add module in front

* Configuration de la relation module - groupe de sites

* Filtre par module pour la requêtes de récupération des groupes de sites

* Add type util module

* Création table cor_sites_group_module et migration des données

* add defaut id_module value

* remov view only on module relationship

* Prevent creating module from group site form

* Remove cache

* Possibilité d'associer un site a un groupe de site depuis un module

* Add Subject for datatable filters instead of inputs

* Affichage nom module dans les routes sites_group

* Ne pas vérifier id_module si liste d'entier

* Application des filtres par type d'objet pour requete geometrique

---------

Co-authored-by: TheoLechemia <[email protected]>
Co-authored-by: amandine-sahl <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants