Last value Cache - Increase in outliers beyond 16 threads for 100 series

[MaduMitha-Ravi]

Increase in Outliers beyond 16 thread concurrency (Last value cache)

• For concurrency threads greater than 16, we are observing more outliers nearly 5x-10x of the typical latency thus impacting the P95 numbers
• CPU usage was less than 20% and Memory consumption was less than 25%
• This observation and pattern looks like there are some restrictions/limitations that are paving way for outliers of latencies.

Could there be a wait happening on some internal resources?

Evidence

Note: How we capture latency (P95 reported) is by having backgrounded threads which are 12, 14, 16 etc. and collect the metrics from just one. This shows on concurrent load, how a particular user observes performance. Stating that, QPS could have been impacted by the outliers observed.