Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Safety in RL #101

Open
2 tasks done
Webbah opened this issue Jun 22, 2023 · 4 comments
Open
2 tasks done

Safety in RL #101

Webbah opened this issue Jun 22, 2023 · 4 comments
Assignees
@Webbah
Labels
enhancement New feature or request

Comments

@Webbah
Copy link
Contributor

Webbah commented Jun 22, 2023 •
edited
Loading

@Webbah Webbah added the enhancement New feature or request label Jun 22, 2023
@Webbah Webbah self-assigned this Jun 22, 2023
Webbah added a commit that referenced this issue Jul 13, 2023
@Webbah
Safety in RL #101 started with condensed matrices
557c32c
Webbah added a commit that referenced this issue Jul 13, 2023
@Webbah
Safety in RL #101 added polytop lib
ac45d78
Webbah added a commit that referenced this issue Jul 14, 2023
@Webbah
Safety in RL #101 added condensed matrices
dc1e46e
Webbah added a commit that referenced this issue Jul 21, 2023
@Webbah
Safety in RL #101
87ce439 
- feasible set similar results compared to matlab and python polyhedra tools
Webbah added a commit that referenced this issue Jul 21, 2023
@Webbah
Safety in RL #101
1d5459c 
- shifted polytope calculation functions
@Webbah
Copy link
Contributor Author

Webbah commented Jul 26, 2023 •
edited
Loading

Proof of concept:
Delivers the same results for a known (P10) parameter setting) then Matlab and python do.

Voltage:
Matlab:
grafik

Julia:
grafik

Current:
Matlab:
grafik

Julia:
grafik

@Webbah
Copy link
Contributor Author

Webbah commented Jul 26, 2023 •
edited
Loading

Remaining problems:

volume calculation of projected polyhedron (eliminate(poly, [4, 5])) gives 0 in 3 dimensions - why?

Maybe problem with polyheron definition on our side?
But the same polyhedron prjected to 2 dimensions has a volume

Next steps:

  • Include safeguard into ElectricGrid framework

Webbah added a commit that referenced this issue Jul 26, 2023
@Webbah
Safety in RL #101
69e4162 
- first working safeguard with similar results to matlab and python
Webbah added a commit that referenced this issue Jul 26, 2023
@Webbah
Safety in RL #101 similar problem with projection
3ab5acf
@Webbah
Copy link
Contributor Author

Webbah commented Aug 25, 2023

Proof of concept ready which can be used in future work to integrate the safeguard into the toolbox

@Webbah Webbah closed this as completed Aug 25, 2023
VikasChidananda added a commit that referenced this issue Sep 7, 2023
@VikasChidananda
#101 Add tests for other voltages
0c7f037
@Webbah
Copy link
Contributor Author

Webbah commented Sep 27, 2023

Stay open since the issue is open for future work

@Webbah Webbah reopened this Sep 27, 2023
janstenner pushed a commit that referenced this issue Sep 29, 2023
@VikasChidananda
Feature 102 extended unit tests (#126)
652e655 
* #101 Add tests for other voltages

* #45 extended tests

* delete `RL_tests`
janstenner added a commit that referenced this issue Sep 29, 2023
@janstenner @VikasChidananda
@Webbah @MarvinMeyer @benebrueck @wallscheid
Small updates (#128)
d94731c 
* Feature 100 additional rl (#110)

* initital file to start working with

* bugfixes

* update

* #100 add td3 agent

* #100 add test scripts

* #100 convert action space using wrappers

* #100 add experiments with td3

* fix `TD3Policy` not defined

* adapt td3 to multi dims action and state

* #100 log first results with TD3

* #100 add wandb support

* #100 add notebook example for `TD3`

* #100 fix `max_steps` not defined

* #100 re organise

* fix spelling

* #100 add wandb logged plots

* add wandb, statistcs, revise to project toml

* #100 rm wandb

* #100 fix multiple methods with same name

* #100 fix infos

* #100 rm Wandb

* #100 fix minor issues

* #100 add `wandb` example

* #100 update with figures

* #100 remove `@show`

---------

Co-authored-by: Jan Stenner <[email protected]>

* initial GUI commit

* Feature 23 modeling variable dc link voltage (#118)

* #23 Save for merge.

* #23 Add battery notebook.

* #23 Start plotting the data.

* Added Plotly.js

* Adjusting the notebook and src to the current version.

* Changed function within PV arrays. Added battery model. Changed DEMO PV to DC link models.

* Commiting current state of the notebook. Implementation to Env is still missing.

* Implementing variable dc link voltage.

* Include variable_dc_link.jl

* Deleted Prints and added stuff to nb

* Updated parameter check. Added default values for battery and pv.

* Changed temperature offset to match the one in the battery.

* Finalization of DC Link voltages, adding two example scripts and one example notebook.

* Fixed bug in the pv array

* Update project.toml

* Added check for the source_type key.

* Corrected calculation of DC link voltages.

* #118 Corrected typos

* Cleaned up code. Added calculation of v_dc to env.reset.

* Cleared notebook and added figures.

* Revert "Merged changes."

This reverts commit 2d20e95, reversing
changes made to 9d25c2c.

---------

Co-authored-by: Marvin Meyer <[email protected]>
Co-authored-by: webbah <[email protected]>

* Feature 96 non linear modelling (#120)

* small test file non linear

* test_non_linear

* added Nodconstructor test for nonlinear things

* update

* smal Test

* update: same result as in the nodeconstructor

* update

* update

* update

* small update

* another try

* NodeConstructor works with functions

* Update:
- node_constructor.jl
- electric_grid_env.jl
Both work with nonlinearity

* Small changed using DifferentialEqautions.jl

* small updates

* Update Env

* update: env_example_nonlinear.jl

* updated env_example_nonlinear.jl
and custom_crontrol.jl to get the last x value. Nothing in the middle

* Changed the accuracy of the ODE solver to speed up

* minor changes

* Info for the solver

* Updated Info, and small Notebook

* Reexport added

* merge ended

* Toml update

* fixed nameof bug and updated the Project.toml

* Deleted unnecessary things

* Updated Nonlinear.ipynb

---------

Co-authored-by: Jan Stenner <[email protected]>

* gui update

* mini gui fix

* import Interpolations.LinearInterpolation

* docs update 1

* docs update 2

* RL example preparation 2

* RL example updates

* Changed dtype of the matrix to Float. (#121)

* RL update

* added fixes and osc feature from RL Classic branch

* updates to RL Single Example (also notebook&docs)

* Feature 96 non linear modelling (#123)

* small test file non linear

* test_non_linear

* added Nodconstructor test for nonlinear things

* update

* smal Test

* update: same result as in the nodeconstructor

* update

* update

* update

* small update

* another try

* NodeConstructor works with functions

* Update:
- node_constructor.jl
- electric_grid_env.jl
Both work with nonlinearity

* Small changed using DifferentialEqautions.jl

* small updates

* Update Env

* update: env_example_nonlinear.jl

* updated env_example_nonlinear.jl
and custom_crontrol.jl to get the last x value. Nothing in the middle

* Changed the accuracy of the ODE solver to speed up

* minor changes

* Info for the solver

* Updated Info, and small Notebook

* Reexport added

* merge ended

* Toml update

* fixed nameof bug and updated the Project.toml

* Deleted unnecessary things

* Updated Nonlinear.ipynb

* changes Notebook

* Updated notebook

* Update Project.toml

---------

Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Jan Stenner <[email protected]>
Co-authored-by: Jan Stenner <[email protected]>

* cleanup: Nonlinear Notebook and RLtests

Docs update: New Nonlinear contents and fixed a bug with multiple plots per page ("gd" to "gdd" etc.) BY HAND

* cleanup

* cleanup and minor fixes again

* Update README.md (#127)

correct typo

* Feature 102 extended unit tests (#126)

* #101 Add tests for other voltages

* #45 extended tests

* delete `RL_tests`

---------

Co-authored-by: Vikas <[email protected]>
Co-authored-by: webbah <[email protected]>
Co-authored-by: MarvinMeyer <[email protected]>
Co-authored-by: Marvin Meyer <[email protected]>
Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Oliver Wallscheid <[email protected]>
Webbah added a commit that referenced this issue Oct 5, 2023
@janstenner @VikasChidananda
@Webbah @MarvinMeyer @benebrueck @wallscheid @SeptimusBoshoff
Newest version into main (#134)
03fbcc0 
* Feature 100 additional rl (#110)

* initital file to start working with

* bugfixes

* update

* #100 add td3 agent

* #100 add test scripts

* #100 convert action space using wrappers

* #100 add experiments with td3

* fix `TD3Policy` not defined

* adapt td3 to multi dims action and state

* #100 log first results with TD3

* #100 add wandb support

* #100 add notebook example for `TD3`

* #100 fix `max_steps` not defined

* #100 re organise

* fix spelling

* #100 add wandb logged plots

* add wandb, statistcs, revise to project toml

* #100 rm wandb

* #100 fix multiple methods with same name

* #100 fix infos

* #100 rm Wandb

* #100 fix minor issues

* #100 add `wandb` example

* #100 update with figures

* #100 remove `@show`

---------

Co-authored-by: Jan Stenner <[email protected]>

* initial GUI commit

* Feature 23 modeling variable dc link voltage (#118)

* #23 Save for merge.

* #23 Add battery notebook.

* #23 Start plotting the data.

* Added Plotly.js

* Adjusting the notebook and src to the current version.

* Changed function within PV arrays. Added battery model. Changed DEMO PV to DC link models.

* Commiting current state of the notebook. Implementation to Env is still missing.

* Implementing variable dc link voltage.

* Include variable_dc_link.jl

* Deleted Prints and added stuff to nb

* Updated parameter check. Added default values for battery and pv.

* Changed temperature offset to match the one in the battery.

* Finalization of DC Link voltages, adding two example scripts and one example notebook.

* Fixed bug in the pv array

* Update project.toml

* Added check for the source_type key.

* Corrected calculation of DC link voltages.

* #118 Corrected typos

* Cleaned up code. Added calculation of v_dc to env.reset.

* Cleared notebook and added figures.

* Revert "Merged changes."

This reverts commit 2d20e95, reversing
changes made to 9d25c2c.

---------

Co-authored-by: Marvin Meyer <[email protected]>
Co-authored-by: webbah <[email protected]>

* Feature 96 non linear modelling (#120)

* small test file non linear

* test_non_linear

* added Nodconstructor test for nonlinear things

* update

* smal Test

* update: same result as in the nodeconstructor

* update

* update

* update

* small update

* another try

* NodeConstructor works with functions

* Update:
- node_constructor.jl
- electric_grid_env.jl
Both work with nonlinearity

* Small changed using DifferentialEqautions.jl

* small updates

* Update Env

* update: env_example_nonlinear.jl

* updated env_example_nonlinear.jl
and custom_crontrol.jl to get the last x value. Nothing in the middle

* Changed the accuracy of the ODE solver to speed up

* minor changes

* Info for the solver

* Updated Info, and small Notebook

* Reexport added

* merge ended

* Toml update

* fixed nameof bug and updated the Project.toml

* Deleted unnecessary things

* Updated Nonlinear.ipynb

---------

Co-authored-by: Jan Stenner <[email protected]>

* gui update

* mini gui fix

* import Interpolations.LinearInterpolation

* docs update 1

* docs update 2

* RL example preparation 2

* RL example updates

* Changed dtype of the matrix to Float. (#121)

* RL update

* added fixes and osc feature from RL Classic branch

* updates to RL Single Example (also notebook&docs)

* Feature 96 non linear modelling (#123)

* small test file non linear

* test_non_linear

* added Nodconstructor test for nonlinear things

* update

* smal Test

* update: same result as in the nodeconstructor

* update

* update

* update

* small update

* another try

* NodeConstructor works with functions

* Update:
- node_constructor.jl
- electric_grid_env.jl
Both work with nonlinearity

* Small changed using DifferentialEqautions.jl

* small updates

* Update Env

* update: env_example_nonlinear.jl

* updated env_example_nonlinear.jl
and custom_crontrol.jl to get the last x value. Nothing in the middle

* Changed the accuracy of the ODE solver to speed up

* minor changes

* Info for the solver

* Updated Info, and small Notebook

* Reexport added

* merge ended

* Toml update

* fixed nameof bug and updated the Project.toml

* Deleted unnecessary things

* Updated Nonlinear.ipynb

* changes Notebook

* Updated notebook

* Update Project.toml

---------

Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Jan Stenner <[email protected]>
Co-authored-by: Jan Stenner <[email protected]>

* cleanup: Nonlinear Notebook and RLtests

Docs update: New Nonlinear contents and fixed a bug with multiple plots per page ("gd" to "gdd" etc.) BY HAND

* cleanup

* cleanup and minor fixes again

* Update README.md (#127)

correct typo

* Feature 102 extended unit tests (#126)

* #101 Add tests for other voltages

* #45 extended tests

* delete `RL_tests`

* changes done Sep oliver (#132)

Co-authored-by: SeptimusBoshoff <[email protected]>

* updated two RL scripts

* Cleanup with Daniel and Marvin

* Hotfix example scripts (#133)

* Update test_battery.

* Added comments and formatting to scripts.

* Deleted DEMO from nb names

* Changed the plotting function

* Renaming the DC Link example scripts

* readme cleanups and RL_example restore

* Changed names within the NB (#135)

* typo and formatting corrections

---------

Co-authored-by: Vikas <[email protected]>
Co-authored-by: webbah <[email protected]>
Co-authored-by: MarvinMeyer <[email protected]>
Co-authored-by: Marvin Meyer <[email protected]>
Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Oliver Wallscheid <[email protected]>
Co-authored-by: SeptimusBoshoff <[email protected]>
Co-authored-by: Wallscheid <[email protected]>
janstenner added a commit that referenced this issue Oct 5, 2023
@janstenner @VikasChidananda
@Webbah @MarvinMeyer @benebrueck @wallscheid @SeptimusBoshoff
Develop (#136)
3057471 
* Feature 100 additional rl (#110)

* initital file to start working with

* bugfixes

* update

* #100 add td3 agent

* #100 add test scripts

* #100 convert action space using wrappers

* #100 add experiments with td3

* fix `TD3Policy` not defined

* adapt td3 to multi dims action and state

* #100 log first results with TD3

* #100 add wandb support

* #100 add notebook example for `TD3`

* #100 fix `max_steps` not defined

* #100 re organise

* fix spelling

* #100 add wandb logged plots

* add wandb, statistcs, revise to project toml

* #100 rm wandb

* #100 fix multiple methods with same name

* #100 fix infos

* #100 rm Wandb

* #100 fix minor issues

* #100 add `wandb` example

* #100 update with figures

* #100 remove `@show`

---------

Co-authored-by: Jan Stenner <[email protected]>

* initial GUI commit

* Feature 23 modeling variable dc link voltage (#118)

* #23 Save for merge.

* #23 Add battery notebook.

* #23 Start plotting the data.

* Added Plotly.js

* Adjusting the notebook and src to the current version.

* Changed function within PV arrays. Added battery model. Changed DEMO PV to DC link models.

* Commiting current state of the notebook. Implementation to Env is still missing.

* Implementing variable dc link voltage.

* Include variable_dc_link.jl

* Deleted Prints and added stuff to nb

* Updated parameter check. Added default values for battery and pv.

* Changed temperature offset to match the one in the battery.

* Finalization of DC Link voltages, adding two example scripts and one example notebook.

* Fixed bug in the pv array

* Update project.toml

* Added check for the source_type key.

* Corrected calculation of DC link voltages.

* #118 Corrected typos

* Cleaned up code. Added calculation of v_dc to env.reset.

* Cleared notebook and added figures.

* Revert "Merged changes."

This reverts commit 2d20e95, reversing
changes made to 9d25c2c.

---------

Co-authored-by: Marvin Meyer <[email protected]>
Co-authored-by: webbah <[email protected]>

* Feature 96 non linear modelling (#120)

* small test file non linear

* test_non_linear

* added Nodconstructor test for nonlinear things

* update

* smal Test

* update: same result as in the nodeconstructor

* update

* update

* update

* small update

* another try

* NodeConstructor works with functions

* Update:
- node_constructor.jl
- electric_grid_env.jl
Both work with nonlinearity

* Small changed using DifferentialEqautions.jl

* small updates

* Update Env

* update: env_example_nonlinear.jl

* updated env_example_nonlinear.jl
and custom_crontrol.jl to get the last x value. Nothing in the middle

* Changed the accuracy of the ODE solver to speed up

* minor changes

* Info for the solver

* Updated Info, and small Notebook

* Reexport added

* merge ended

* Toml update

* fixed nameof bug and updated the Project.toml

* Deleted unnecessary things

* Updated Nonlinear.ipynb

---------

Co-authored-by: Jan Stenner <[email protected]>

* gui update

* mini gui fix

* import Interpolations.LinearInterpolation

* docs update 1

* docs update 2

* RL example preparation 2

* RL example updates

* Changed dtype of the matrix to Float. (#121)

* RL update

* added fixes and osc feature from RL Classic branch

* updates to RL Single Example (also notebook&docs)

* Feature 96 non linear modelling (#123)

* small test file non linear

* test_non_linear

* added Nodconstructor test for nonlinear things

* update

* smal Test

* update: same result as in the nodeconstructor

* update

* update

* update

* small update

* another try

* NodeConstructor works with functions

* Update:
- node_constructor.jl
- electric_grid_env.jl
Both work with nonlinearity

* Small changed using DifferentialEqautions.jl

* small updates

* Update Env

* update: env_example_nonlinear.jl

* updated env_example_nonlinear.jl
and custom_crontrol.jl to get the last x value. Nothing in the middle

* Changed the accuracy of the ODE solver to speed up

* minor changes

* Info for the solver

* Updated Info, and small Notebook

* Reexport added

* merge ended

* Toml update

* fixed nameof bug and updated the Project.toml

* Deleted unnecessary things

* Updated Nonlinear.ipynb

* changes Notebook

* Updated notebook

* Update Project.toml

---------

Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Jan Stenner <[email protected]>
Co-authored-by: Jan Stenner <[email protected]>

* cleanup: Nonlinear Notebook and RLtests

Docs update: New Nonlinear contents and fixed a bug with multiple plots per page ("gd" to "gdd" etc.) BY HAND

* cleanup

* cleanup and minor fixes again

* Update README.md (#127)

correct typo

* Feature 102 extended unit tests (#126)

* #101 Add tests for other voltages

* #45 extended tests

* delete `RL_tests`

* changes done Sep oliver (#132)

Co-authored-by: SeptimusBoshoff <[email protected]>

* updated two RL scripts

* Cleanup with Daniel and Marvin

* Hotfix example scripts (#133)

* Update test_battery.

* Added comments and formatting to scripts.

* Deleted DEMO from nb names

* Changed the plotting function

* Renaming the DC Link example scripts

* readme cleanups and RL_example restore

* Changed names within the NB (#135)

* typo and formatting corrections

* Fixed typo in README, set new GUI window name

---------

Co-authored-by: Vikas <[email protected]>
Co-authored-by: webbah <[email protected]>
Co-authored-by: MarvinMeyer <[email protected]>
Co-authored-by: Marvin Meyer <[email protected]>
Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Benedikt Brück <[email protected]>
Co-authored-by: Oliver Wallscheid <[email protected]>
Co-authored-by: SeptimusBoshoff <[email protected]>
Co-authored-by: Wallscheid <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Webbah Webbah

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Webbah