Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge 3.6.x into main #12996

Closed
wants to merge 24 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
0d58385
Remove Snyk jobs from CI scanning
rasa-jmac Jul 12, 2023
dc6f00d
Fix line formatting
rasa-jmac Jul 12, 2023
58e122f
Made use of drop-small-last-batch logic only possible in DIET and con…
twerkmeister Nov 16, 2023
048dc80
adjusted release script to work with public remote (#12953)
twerkmeister Nov 17, 2023
241bf28
Prepare release 3.6.14 (#12956)
twerkmeister Nov 20, 2023
609287e
Port "connection timeout to action server" changes to 3.6.x - [ENG 68…
varunshankar Nov 30, 2023
7cfc7d0
prepared release of version 3.6.15 (#12967)
varunshankar Nov 30, 2023
5d037b9
Merge branch '3.6.x' into security/remove_snyk_ci_scans
rasa-jmac Dec 11, 2023
ff52994
Merge pull request #12667 from RasaHQ/security/remove_snyk_ci_scans
rasa-jmac Dec 11, 2023
865902f
Additional load testing recommendations
sanchariGr Dec 11, 2023
e7a38d9
Merge branch '3.6.x' into ATO-1628
sanchariGr Dec 11, 2023
ecdd8be
[ENG-712] Fix dependency install failures on windows (#150)
tabergma Dec 21, 2023
96eead7
update cryptography
Tawakalt Dec 29, 2023
45cbefd
add changelog
Tawakalt Jan 2, 2024
559aa39
Merge branch '3.6.x' into ATO-1959-patch-security-vulnerability-3.6.x
Tawakalt Jan 2, 2024
b1199ca
Merge pull request #12983 from RasaHQ/ATO-1959-patch-security-vulnera…
Tawakalt Jan 2, 2024
652c175
Review changes made
sanchariGr Jan 4, 2024
9cafd12
Merge branch '3.6.x' into ATO-1628
sanchariGr Jan 4, 2024
64546fc
Add missing CI step condition
sanchariGr Jan 5, 2024
fc72dd8
Merge pull request #12974 from RasaHQ/ATO-1628
sanchariGr Jan 5, 2024
e970b3a
update changelog with entries from 3.5.x releases
m-vdb Jan 16, 2024
6019299
Merge pull request #12995 from RasaHQ/update-changelog-from-3.5
m-vdb Jan 16, 2024
5f451fc
Merge branch 'main' into merge-3.6.x-main-6019299
sanchariGr Jan 17, 2024
1b7256b
Update sonar github actions
sanchariGr Jan 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion .github/workflows/continous-integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,13 @@ jobs:
run: |
poetry self add setuptools

- name: Prevent race condition in poetry build
# More context about race condition during poetry build can be found here:
# https://github.com/python-poetry/poetry/issues/7611#issuecomment-1747836233
if: needs.changes.outputs.backend == 'true'
run: |
poetry config installer.max-workers 1

- name: Load Poetry Cached Libraries ⬇
id: cache-poetry
if: needs.changes.outputs.backend == 'true'
Expand Down Expand Up @@ -589,7 +596,7 @@ jobs:
coverage xml

- name: Analyse code with SonarCloud
uses: sonarsource/sonarcloud-github-action@5875562561d22a34be0c657405578705a169af6c
uses: sonarsource/sonarcloud-github-action@49e6cd3b187936a73b8280d59ffd9da69df63ec9
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
Expand Down
21 changes: 0 additions & 21 deletions .github/workflows/security-scans.yml
Original file line number Diff line number Diff line change
Expand Up @@ -129,24 +129,3 @@ jobs:
- name: Run Bandit 🔪
if: needs.changes.outputs.backend == 'true'
run: make lint-security

snyk:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c
- name: Run Snyk Open Source to check for Python vulnerabilities
uses: snyk/actions/python-3.8@master
continue-on-error: true
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
command: monitor
args: --all-projects --org=rasa --skip-unresolved
- name: Run Snyk Open Source to check for JS vulnerabilities
uses: snyk/actions/node@master
continue-on-error: true
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
command: monitor
args: --org=rasa --yarn-workspaces --strict-out-of-sync=false --prune-repeated-subdependencies
115 changes: 89 additions & 26 deletions CHANGELOG.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,24 +16,38 @@ https://github.com/RasaHQ/rasa/tree/main/changelog/ . -->

<!-- TOWNCRIER -->

## [3.6.15] - 2023-11-30

Rasa 3.6.15 (2023-11-30)
### Bugfixes
- [#12965](https://github.com/rasahq/rasa/issues/12965): Fixed connection timeout to action server by setting KEEP_ALIVE_TIMEOUT to 120, and reverting changes introduced in #12886.


## [3.6.14] - 2023-11-17

Rasa 3.6.14 (2023-11-17)
### Bugfixes
- [#12948](https://github.com/rasahq/rasa/issues/12948): Fixed UnexpecTEDIntentlessPolicy training errors that resulted from a change to batching behavior. Changed the batching behavior back to the original for all components. Made the changed batching behavior accessible in DietClassifier using `drop_small_last_batch: True`.


## [3.6.13] - 2023-10-23
Rasa 3.6.13 (2023-10-23)

Rasa 3.6.13 (2023-10-23)
### Bugfixes
- [#12927](https://github.com/rasahq/rasa/issues/12927): Fix wrong conflicts that occur when rasa validate stories is run with slots that have active_loop set to null in mapping conditions.


## [3.6.12] - 2023-10-10
Rasa 3.6.12 (2023-10-10)

Rasa 3.6.12 (2023-10-10)
### Bugfixes
- [#12904](https://github.com/rasahq/rasa/issues/12904): Refresh headers used in requests (e.g. action server requests) made by `EndpointConfig` using its `headers` attribute.
- [#12906](https://github.com/rasahq/rasa/issues/12906): Upgrade `pillow` to `10.0.1` to address security vulnerability CVE-2023-4863 found in `10.0.0` version.


## [3.6.11] - 2023-10-05
Rasa 3.6.11 (2023-10-05)

Rasa 3.6.11 (2023-10-05)
### Bugfixes
- [#12722](https://github.com/rasahq/rasa/issues/12722): Intent names will not be falsely abbreviated in interactive training (fixes OSS-413).

Expand All @@ -45,8 +59,8 @@ Rasa 3.6.11 (2023-10-05)


## [3.6.10] - 2023-09-26
Rasa 3.6.10 (2023-09-26)

Rasa 3.6.10 (2023-09-26)
### Improvements
- [#12827](https://github.com/rasahq/rasa/issues/12827): Improved handling of last batch during DIET and TED training. The last batch is discarded if it contains less than half a batch size of data.
- [#12852](https://github.com/rasahq/rasa/issues/12852): Added `username` to the connection parameters for `RedisLockStore` and `RedisTrackerStore`
Expand All @@ -57,8 +71,8 @@ Rasa 3.6.10 (2023-09-26)


## [3.6.9] - 2023-09-15
Rasa 3.6.9 (2023-09-15)

Rasa 3.6.9 (2023-09-15)
### Improvements
- [#12778](https://github.com/rasahq/rasa/issues/12778): Added additional method `fingerprint_addon` to the `GraphComponent` interface to allow inclusion of external data into the fingerprint calculation of a component

Expand All @@ -67,29 +81,29 @@ Rasa 3.6.9 (2023-09-15)


## [3.6.8] - 2023-08-30
Rasa 3.6.8 (2023-08-30)

Rasa 3.6.8 (2023-08-30)

No significant changes.


## [3.6.7] - 2023-08-29
Rasa 3.6.7 (2023-08-29)

Rasa 3.6.7 (2023-08-29)
### Bugfixes
- [#12768](https://github.com/rasahq/rasa/issues/12768): Updated certifi, cryptography, and scipy packages to address security vulnerabilities.


## [3.6.6] - 2023-08-23
Rasa 3.6.6 (2023-08-23)

Rasa 3.6.6 (2023-08-23)
### Bugfixes
- [#12755](https://github.com/rasahq/rasa/issues/12755): Updated setuptools and wheel to address security vulnerabilities.


## [3.6.5] - 2023-08-17
Rasa 3.6.5 (2023-08-17)

Rasa 3.6.5 (2023-08-17)
### Improvements
- [#12696](https://github.com/rasahq/rasa/issues/12696): Use the same session across requests in `RasaNLUHttpInterpreter`

Expand All @@ -102,8 +116,8 @@ Rasa 3.6.5 (2023-08-17)


## [3.6.4] - 2023-07-21
Rasa 3.6.4 (2023-07-21)

Rasa 3.6.4 (2023-07-21)
### Bugfixes
- [#12575](https://github.com/rasahq/rasa/issues/12575): Extract conditional response variation and channel variation filtering logic into a separate component.
Enable usage of this component in the NaturalLanguageGenerator subclasses (e.g. CallbackNaturalLanguageGenerator, TemplatedNaturalLanguageGenerator).
Expand All @@ -114,8 +128,8 @@ Rasa 3.6.4 (2023-07-21)


## [3.6.3] - 2023-07-20
Rasa 3.6.3 (2023-07-20)

Rasa 3.6.3 (2023-07-20)
### Improvements
- [#12637](https://github.com/rasahq/rasa/issues/12637): Added a human readable component to structlog using the `event_info` key and made it the default rendered key if present.

Expand All @@ -130,15 +144,15 @@ Rasa 3.6.3 (2023-07-20)


## [3.6.2] - 2023-07-06
Rasa 3.6.2 (2023-07-06)

Rasa 3.6.2 (2023-07-06)
### Bugfixes
- [#12602](https://github.com/rasahq/rasa/issues/12602): Resolves the issue of importing TensorFlow on Docker for ARM64 architecture.


## [3.6.1] - 2023-07-03
Rasa 3.6.1 (2023-07-03)

Rasa 3.6.1 (2023-07-03)
### Improvements
- [#12533](https://github.com/rasahq/rasa/issues/12533): Add building multi-platform Docker image (amd64/arm64)
- [#12543](https://github.com/rasahq/rasa/issues/12543): Switch struct log to `FilteringBoundLogger` in order to retain log level set in the config.
Expand Down Expand Up @@ -205,6 +219,55 @@ Rasa 3.6.0 (2023-06-14)
### Miscellaneous internal changes
- [#12291](https://github.com/rasahq/rasa/issues/12291), [#12329](https://github.com/rasahq/rasa/issues/12329), [#12332](https://github.com/rasahq/rasa/issues/12332), [#12365](https://github.com/rasahq/rasa/issues/12365), [#12372](https://github.com/rasahq/rasa/issues/12372), [#12386](https://github.com/rasahq/rasa/issues/12386), [#12492](https://github.com/rasahq/rasa/issues/12492)

## [3.5.17] - 2023-12-05

Rasa 3.5.17 (2023-12-05)
### Improvements
- [#12851](https://github.com/rasahq/rasa/issues/12851): Added `username` to the connection parameters for `RedisLockStore` and `RedisTrackerStore`
- [#1493](https://github.com/rasahq/rasa/issues/1493): Telemetry data is only send for licensed users.


## [3.5.16] - 2023-08-30

Rasa 3.5.16 (2023-08-30)

No significant changes.


## [3.5.15] - 2023-07-21

Rasa 3.5.15 (2023-07-21)

No significant changes.


## [3.5.14] - 2023-07-12

Rasa 3.5.14 (2023-07-12)
### Bugfixes
- [#12639](https://github.com/rasahq/rasa/issues/12639): Fix the issue with the most recent model not being selected if the owner or permissions where modified on the model file.

### Miscellaneous internal changes
- [#12649](https://github.com/rasahq/rasa/issues/12649)


## [3.5.13] - 2023-07-05

Rasa 3.5.13 (2023-07-05)
### Bugfixes
- [#12549](https://github.com/rasahq/rasa/issues/12549): Introduce a validation step in `rasa data validate` command to identify non-existent paths and empty domains.


## [3.5.12] - 2023-06-23

Rasa 3.5.12 (2023-06-23)
### Bugfixes
- [#12534](https://github.com/rasahq/rasa/issues/12534): Rich responses containing buttons with parentheses characters are now correctly parsed.
Previously any characters found between the first identified pair of `()` in response button took precedence.

### Miscellaneous internal changes
- [#12512](https://github.com/rasahq/rasa/issues/12512)


## [3.5.12] - 2023-06-23

Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ prepare-tests-windows:
# It will retry the installation 5 times if it fails
# See: https://github.com/actions/virtual-environments/blob/main/images/win/scripts/ImageHelpers/ChocoHelpers.ps1
prepare-tests-windows-gha:
powershell -command "Choco-Install wget graphviz"
powershell -command "Install-ChocoPackage wget graphviz"

test: clean
# OMP_NUM_THREADS can improve overall performance using one thread by process (on tensorflow), avoiding overload
Expand Down
1 change: 1 addition & 0 deletions changelog/12983.bugfix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Upgrade Cryptography to fix improper certificate validation.
1 change: 1 addition & 0 deletions changelog/712.misc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Prevent race condition in poetry build to fix dependency install failures on windows.
14 changes: 14 additions & 0 deletions docs/docs/monitoring/load-testing-guidelines.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,26 @@ In order to gather metrics on our system's ability to handle increased loads and
In each test case we spawned the following number of concurrent users at peak concurrency using a [spawn rate](https://docs.locust.io/en/1.5.0/configuration.html#all-available-configuration-options) of 1000 users per second.
In our tests we used the Rasa [HTTP-API](https://rasa.com/docs/rasa/pages/http-api) and the [Locust](https://locust.io/) open source load testing tool.


| Users | CPU | Memory |
|--------------------------|----------------------------------------------|---------------|
| Up to 50,000 | 6vCPU | 16 GB |
| Up to 80,000 | 6vCPU, with almost 90% CPU usage | 16 GB |


### Some recommendations to improve latency
- Sanic Workers must be mapped 1:1 to CPU for both Rasa Pro and Rasa Action Server
- Create `async` actions to avoid any blocking I/O
- `enable_selective_domain: true` : Domain is only sent for actions that needs it. This massively trims the payload between the two pods.
- Consider using compute efficient machines on cloud which are optimized for high performance computing such as the C5 instances on AWS.
However, as they are low on memory, models need to be trained lightweight.


| Machine | RasaPro | Rasa Action Server |
|--------------------------------|------------------------------------------------|--------------------------------------------------|
| AWS C5 or Azure F or Gcloud C2 | 3-7vCPU, 10-16Gb Memory, 3-7 Sanic Threads | 3-7vCPU, 2-12Gb Memory, 3-7 Sanic Threads |


### Debugging bot related issues while scaling up

To test the Rasa [HTTP-API](https://rasa.com/docs/rasa/pages/http-api) ability to handle a large number of concurrent user activity we used the Rasa Pro [tracing](./tracing.mdx) capability
Expand Down
51 changes: 26 additions & 25 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading