Retrying Requests

Retrying Task/Upstream requests

The Concurrent execution, Dispatch-Compose feature provides execution timeouts that may be configured by the developer for all async task executions - such as invoking an upstream service. An upstream service's tail latencies are very likely dependent on transient phenomena like GC that lasts a short duration during which any request served by that service node will experience higher latencies.

GJEX supports retrying requests that may potentially timeout using Request Hedging as described here.

Enabling Request Hedging

Retries can be enabled by using the withRequestHedging = true attribute on @ConcurrentTask(..) annotation as shown below:

@ConcurrentTask(completion=ConcurrentTask.Completion.Optional, timeout = 300, withRequestHedging = true)
public Future<ResponseEntity> tracedMethod3(ResponseEntity entity) {
.......
}

We consider the rolling 95 percentile latency over a 10 second window to determine the actual timeout(which may be still smaller in case a deadline is present) and hedge the request i.e. fire it again and use that response instead.

Caveat: This feature is currently in active development and there aren't benchmarks yet to prove reduction in tail latencies when Request hedging is enabled.