Cloudflare Workers unreliable due to Discord's Cloudflare bans (easy fix)

[wnelson03]

Description

Brought this issue up in #7137 and many agreed, including Cloudflare employees. though I think the Discord employee misunderstood me, he never replied after clarification given.

Experienced by myself and several others #7040

Cloudflare Workers serverless uses Shared IPs to accomplish their massive scale. There are millions of developers sharing Cloudflare's IPs, and Discord's current configuration of their Cloudflare rate limiting rule makes using serverless unreliable with Discord.

Cloudflare Workers is a popular platform used by Discord bot developers. Cloudflare has a very active presence on Discord, assisting developers with Discord integrations on a daily basis.

Discord documentation recommended developers to use Cloudflare Workers numerous times, which makes this issue more surprising.

Cloudflare has a very strong IP reputation, and they do a great job at mitigating spam and DDoS through their platform. Discord wouldn't be using Cloudflare if it wasn't secure in this manner.

Fix

The IP address 2a06:98c0:3600::103 is used always for Worker requests to other Cloudflare websites (Discord is a Cloudflare website) https://developers.cloudflare.com/fundamentals/reference/http-request-headers/#cf-connecting-ip-in-worker-subrequests (thank you @Skye-31)

Additionally, there is a cf-worker HTTP header which is only permitted use by Cloudflare workers. I tested sending header manually with my normal IP and a Cloudflare WARP IP, the header was stripped both times #7137 (comment)

Steps to Reproduce

Send an API request to Discord from a high-load Cloudflare datacenter location. Here are the Cloudflare datacenters with problems so far:

• GRU (São Paulo, Brazil)
• CDG (Paris, France)
• FRA (Frankfurt, Germany)
• WAW (Warsaw, Poland)
  

The requested URL was https://discord.com/api/oauth2/token to exchange OAuth2 code

Expected Behavior

It would be expected that Discord API requests would succeed, with HTTP 200 code.

Current Behavior

Cloudflare Error 1015 (Global Rate Limit), when requesting the Discord API from Cloudflare datacenters that are high-load on Discord's API.

HTTP 429 code, because of Discord's configuration https://discord.com/developers/docs/topics/rate-limits#invalid-request-limit-aka-cloudflare-bans

Screenshots/Videos

Has occurred at least 1 time per week, for months.
I can be sure our software isn't hitting these limits by itself, it's not that large. Even the total requests during that time period are under the limit, and Brazil is not even in our top locations for traffic to our software. It just so happens a lot of other Discord API developers have apps running in Brazil, it seems.

Client and System Information

Cloudflare Workers
AS13335

IP address: 2a06:98c0:3600::103

[afgiel]

our suggested approach to avoid IP bans from shared environments like Cloudflare Workers is to use a dedicated static egress IP. we will not be making Cloudflare Workers exempt from global rate limits.

[Le0Developer]

Do note that is the wrong Cloudflare product and does not apply to workers.
Closests would be Aegis, but that'd require a special orange-to-orange setup or Enterprise magic.

Needless to say, both products are enterprise only. If you have that much money, you can dedicate a server for proxying traffic instead.

I don't think there is a proper solution for this except special logic that replaces the IP address with the cf-worker header for global ratelimits.

[afgiel]

please consult this documentation https://discord.com/developers/docs/topics/rate-limits#invalid-request-limit-aka-cloudflare-bans

[wnelson03]

our suggested approach to avoid IP bans from shared environments like Cloudflare Workers is to use a dedicated static egress IP. we will not be making Cloudflare Workers exempt from global rate limits.

Thank you for responding. It would be good if Discord looked into a separate, higher threshold for Cloudflare workers. Discord has the ability to add several different rate limiting rules on Cloudflare as an enterprise customer.

Also the cf-worker header is unique to each Workers website, so it's a good property to rate-limit by. That would prevent a few sites on Cloudflare workers getting spammed from affecting everyone on Cloudflare workers as a whole.

[afgiel]

we are investigating using cf-worker as a rate limit key, but it may not be feasible for us. in the meantime a dedicated static IP is our best suggestion.