Concurrency discrepancies

[chgar101]

We have noticed that no matter what we enter in the JMeter script for concurrency, ramp-up time, and hold period it appears to be overwritten by what is in AWS. We then tried directly setting the desired concurrency in the AWS testing console and used a .jmx with the http requests but we are not getting the correct amount of virtual users when we look at the live data feed and also after the results display the math does not add up to the specified number of users set at the beginning of the test. The .jmx being used has a total of six individual https requests and when we set the task count to 1 and concurrency to 10 we get 12 total virtual users so seems like no matter what we input for concurrency in AWS it always produces two virtual users per http request that we add to the .jmx, in this case 6 making it 12 users in total. So it comes down to two virtual users per service, instead of the 10 that was specified and it appears to do two no matter what is inputted for concurrency in AWS. The only way we found to change it was to increase the task count. With a task count of 5 we were able to get 30 total virtual users or 5 per http request but still with the task count of 5 it doesn’t seem to honor the inputted concurrency in AWS. Does anyone have any insight on this?
Is there a specific thread group that is supported from JMeter to AWS so that it does honor the parameters set? Why are we not seeing the correct concurrency number and why it is producing two per http request when task count is 1 even though concurrency is set to 10?

[G-Lenz]

Hi @chgar101

The way Taurus works with JMeter thread groups is that it uses the concurrency defined in the YAML (which comes from the UI input) and divides it proportionally by the concurrency provided in the JMeter thread groups. You can see more information in the documentation: https://gettaurus.org/docs/JMeter/#Run-Existing-JMX-File.

We currently have it in our backlog to provide a way to ignore the UI configuration so that it will simply use the JMeter configuration.

[chgar101]

@G-Lenz thanks for the reply. I ran a load test in AWS make sure I understand this and I did not get what I expected. I set the concurrency in JMeter to 10 and then in AWS to 100 but still looks like it just overrides the JMeter settings completely since I see 100 virtual users in the live data. Doesn’t look to be dividing the concurrency from AWS by what is set in JMeter. I used the Concurrency Thread Group in JMeter, is that possibly not supported. Below is a screenshot of the live data results as the test ran.

[G-Lenz]

@chgar101

How many thread groups are you using in JMeter?

[chgar101]

@G-Lenz Only one thread group and have six http requests under it.

[jayfry1077]

Any more info on this would be appreciated.

In the AWS UI I set task count to 1 and concurrency to 1. In my Jmeter thread group I set users to 1 and I have my loop count set to 1.

Running locally my test only runs 1 time.
Running with DLT my test runs over 800 times

I really just can't figure out how i'm supposed to have control over the total number of executions

[kamyarz-aws]

@jayfry1077 The number of executions are determined by the number of tasks, the conurrency of these tasks (Number of virtual Users) and if there is a hold for period that also adds to the number of executions. In a jmeter thread group, the tests runs as many times as you sepcify as it is different here since we attempt to create a higher level of througput.

[andromeda306]

I recently posted a question here but I think it could be suitably answered in the context of this thread as well #151. I too am somewhat confused by the configuration requirements. Should we ignore jmeter thread group settings altogether?

[kamyarz-aws]

Hi @andromeda306

https://docs.aws.amazon.com/solutions/latest/distributed-load-testing-on-aws/determine-number-of-users.html
https://guide.blazemeter.com/hc/en-us/articles/360000864389-Calibrating-a-Taurus-Test-Calibrating-a-Taurus-Test

These could to some extent answer your question.

The solution uses Taurus which runs JMeter. The users are specified in the Taurus test configuration as well as their JMeter script. The way that Taurus works with JMeter thread groups is that when users are defined in both it use the Taurus defined users as the actual number and the JMeter users in each thread group as a proportional factor of the actual. For example if you define two JMeter thread groups with 100 and 200 users respectively, and define 75 users in the DLT solution (which is creating the Taurus configuration), the actual number of users run in each thread group will be 25 and 50. You can read more in the taurus documentation.

[mcawilcox]

Any more info on this would be appreciated.
Running locally my test only runs 1 time. Running with DLT my test runs over 800 times

I really just can't figure out how i'm supposed to have control over the total number of executions

Earlier answers showed you how to have control over the number of executions, but you were likely already in control ... but that the problem was DLT was displaying the wrong number in the graph. The 800 was likely right for number of successful requests, but wrong for the number of virtual users.

This answer inspired me to investigate a little further, and I've just added a bug report for this issue in #206