compute response.ok_pct correctly and add latency metrics for scenarios #4
Conversation
| @@ -12,7 +12,7 @@ | |||
| "license": "Apache-2.0", | |||
| "dependencies": { | |||
| "debug": "2.2.0", | |||
| "datadog-metrics": "^0.3.0" | |||
| "datadog-metrics": "^0.8.1" | |||
There was a problem hiding this comment.
at least 0.8.1 is required to make sure
- count is used instead of counter as counter is deprecated in datadog
- count is not incremented for value 0
| @@ -18,7 +18,7 @@ class DatadogPlugin | |||
| @ee.on 'done', @flushStats | |||
|
|
|||
| getDatadogConfig: -> | |||
| flushIntervalSeconds: 0 | |||
| # flushIntervalSeconds: 0 | |||
There was a problem hiding this comment.
letting auto flush take over since that is how it was even with flushIntervalSeconds = 0 with v0.3.0 of datadog-metrics. datadog-metrics had a bug which was doing auto flush for flushIntervalSeconds = 0, but is now taken care of since 0.5.0.
There was a problem hiding this comment.
Identified a potential issue with "the last flush" - Artillery exits before it can be sent? For example, if the test is ~5 seconds, nothing gets sent to Datadog. This is true for both version 0.1.1 and the version in this PR. Needs further investigation.
| 'response.2xx': [0, datadog.increment] | ||
| 'response.3xx': [0, datadog.increment] | ||
| 'response.4xx': [0, datadog.increment] | ||
| 'response.5xx': [0, datadog.increment] |
There was a problem hiding this comment.
all these metrics are actually counts, and hence we should treat them as datadog metric type count instead of gauge. it doesn't matter when running from one machine, but becomes important when used via serverless-artillery which uses multiple lambdas. these metrics then do not get combined if the metrics are treated as type gauge instead of type count.
| percentage = (metrics['response.2xx'] + metrics['response.3xx']) * 100 \ | ||
| / metrics['scenarios.completed'] | ||
| percentage = (metrics['response.2xx'][0] + metrics['response.3xx'][0]) \ | ||
| * 100 / metrics['requests.completed'][0] |
There was a problem hiding this comment.
this was making percentage > 100 as artillery is counting 200 and 300 as separate requests even when with in the same scenario.
|
Any update on this? |
|
Hey - thank you for the PR and apologizes for not replying sooner (notifications -> /dev/null?). I will find some time to review & test the changes within a ~week. |
|
Lookin' good |
| @@ -1,3 +1,4 @@ | |||
| node_modules/ | |||
| src/*.js | |||
| test/*.js | |||
| .idea/ | |||
There was a problem hiding this comment.
.idea/ is developer / machine specific, a global .gitignore is preferred.
There was a problem hiding this comment.
Thank you for sharing that.
| @@ -1,6 +1,6 @@ | |||
| { | |||
| "name": "artillery-plugin-datadog", | |||
| "version": "0.1.1", | |||
| "version": "0.1.2", | |||
There was a problem hiding this comment.
Will bump the minor version due to datadog-metrics update and the change from gauge to count.
| @@ -18,7 +18,7 @@ class DatadogPlugin | |||
| @ee.on 'done', @flushStats | |||
|
|
|||
| getDatadogConfig: -> | |||
| flushIntervalSeconds: 0 | |||
| # flushIntervalSeconds: 0 | |||
There was a problem hiding this comment.
Identified a potential issue with "the last flush" - Artillery exits before it can be sent? For example, if the test is ~5 seconds, nothing gets sent to Datadog. This is true for both version 0.1.1 and the version in this PR. Needs further investigation.
|
Thank you for the PR - merged and released Potential data flush issue described in #5, needs further work to investigate. |
No description provided.