Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle Retry-After header #151

Open
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

doyaaaaaken
Copy link
Contributor

Fixes #105 #108

Read Retry-After response header, and decide when to retry next.

This is WIP pull request, so below tasks are not started.

Please review whether the direction is correct.

@codecov
Copy link

codecov bot commented Aug 15, 2019

Codecov Report

Merging #151 into master will increase coverage by 0.08%.
The diff coverage is 95%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #151      +/-   ##
============================================
+ Coverage     90.47%   90.55%   +0.08%     
- Complexity      128      137       +9     
============================================
  Files            42       42              
  Lines           399      413      +14     
  Branches         45       50       +5     
============================================
+ Hits            361      374      +13     
  Misses           14       14              
- Partials         24       25       +1
Impacted Files Coverage Δ Complexity Δ
...rybalkinsd/kohttp/interceptors/RetryInterceptor.kt 93.93% <95%> (-0.8%) 21 <15> (+9)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ac0a9b...c3a0dc8. Read the comment docs.

@rybalkinsd
Copy link
Owner

Hi @doyaaaaaken , I will be able to check this PR on weekend.

Copy link
Owner

@rybalkinsd rybalkinsd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@doyaaaaaken
You're on a right way, please check my comments

private val invocationTimeout: Long = 0,
private val ratio: Int = 1,
private val errorStatuses: List<Int> = listOf(503, 504)
private val failureThreshold: Int = 3,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please avoid formatting change

@@ -34,7 +35,13 @@ class RetryInterceptor(
delay = performAndReturnDelay(delay)
}
val response = chain.proceed(request)
if (!isRetry(response, attemptsCount)) return response

val nextTime = calculateNextRetry(response, attemptsCount, delay, chain.readTimeoutMillis())
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a room for improvement here.

if () statement else return logic looks rather complicated

@@ -34,7 +35,13 @@ class RetryInterceptor(
delay = performAndReturnDelay(delay)
}
val response = chain.proceed(request)
if (!isRetry(response, attemptsCount)) return response

val nextTime = calculateNextRetry(response, attemptsCount, delay, chain.readTimeoutMillis())
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to have two stages here:

  1. check if we're going to retry ?
  2. calculate next delay.

Probably, we need to modify performAndReturnDelay

internal fun isRetry(response: Response, attemptsCount: Int): Boolean =
attemptsCount < failureThreshold && response.code() in errorStatuses
internal fun calculateNextRetry(response: Response, attemptsCount: Int, delay: Long, maxDelayTime: Int): Long? {
val retryAfter = response.header("Retry-After")?.toLongOrNull()
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably need an ext function to get header by name.

BTW, did you check what is the metric of Retry-After values? s/ms ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably need an ext function to get header by name.

OK!

BTW, did you check what is the metric of Retry-After values? s/ms ?

The unit of Retry-After value is seconds. But in my code it is millisecond.
Sorry, I'll fix it.

Copy link
Owner

@rybalkinsd rybalkinsd Aug 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no worries at all, that's why we're doing reviews :)

@rybalkinsd rybalkinsd self-assigned this Aug 27, 2019
@rybalkinsd rybalkinsd added this to the 0.12.0 milestone Aug 27, 2019
@rybalkinsd
Copy link
Owner

as for now, targeting to 0.12.0 with this PR

internal fun isRetry(response: Response, attemptsCount: Int): Boolean =
attemptsCount < failureThreshold && response.code() in errorStatuses
internal fun calculateNextRetry(response: Response, attemptsCount: Int, delay: Long, maxDelayTime: Int): Long? {
val retryAfter = response.header("Retry-After")?.toLongOrNull()?.let { it * 1000 }
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@doyaaaaaken
Copy link
Contributor Author

doyaaaaaken commented Sep 1, 2019

I simplified logic according to this comment (#151 (comment)).

And as I said in the first comment (#151 (comment)), below tasks are already left.

internal fun parseRetryAfter(headers: Headers): Long? {
val retryAfter = headers["Retry-After"] ?: return null
return retryAfter.toLongOrNull()?.let { it * 1000 } ?: try {
val date = httpDateFormat.parse(retryAfter).toInstant().epochSecond
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working with time is tricky. I strongly suggest several things:

  1. Do not do any custom calculations. with timestamps(epoch, etc )

  2. Use joda time primitives. It will take care of all things like Arabic, Japanese, or Thai calendars.

     val responseDate = ZonedDateTime.parse("...")
     val now = ZonedDateTime.now()
     val duration = Duration.between(responseDate, now)
    
  3. Please separate two stratagies in your code: 1st is to parse as number; 2nd is to parse as timestamp

return retryAfter.toLongOrNull()?.let { it * 1000 } ?: try {
val date = httpDateFormat.parse(retryAfter).toInstant().epochSecond
val now = Instant.now().epochSecond
(date - now).takeIf { it > 0 }
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's great that you check it to be positive!

@doyaaaaaken
Copy link
Contributor Author

I fixes code about this comment!
#151 (comment)

@doyaaaaaken doyaaaaaken changed the title [WIP] handle Retry-After header handle Retry-After header Sep 4, 2019
@doyaaaaaken
Copy link
Contributor Author

All fixes are done, so I remove 'WIP'.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Retry headers handling in retry interceptor
2 participants