-
Notifications
You must be signed in to change notification settings - Fork 750
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compatibility between ENABLE_SUBNET_DISCOVERY
and ENABLE_PREFIX_DELEGATION
?
#2959
Comments
There shouldn't be any dependency between these two features. What you are likely running into is this open issue that we have with Enhanced Subnet Discovery - #2908 ; which is being fixed right now - #2954 |
@orsenthil I certainly don't know enough to say otherwise, but reading #2908, the writeup doesn't sound like it's related. My primary subnet is not exhausted, it does have IP space, a /28 is getting assigned, and the pods are created and running. It's just that the /28 is getting assigned from the wrong subnet, one that is not tagged.
Anyway, hopefully we're saying the same thing, and #2954 does indeed resolve my issue also! |
@lorengordon When |
Oh, interesting. No, a secondary ENI is not being created. It's just assigning prefixes to the primary ENI for the node. Maybe this is intended behavior at the moment. The confusing part to me about the behavior in that case would be that However, if I disable |
It sounds like you are trying to force some pods onto a certain subnet, in which Custom Networking would fit that use case better. |
Hmm. Ok, possibly, maybe I'm doing this all wrong, and I need to use the secondary subnets for the node group primary interface?
To the degree that some is a subset of all, yeah.... Yes, I am trying to manage CIDR allocations that are routed across a larger corporate environment, while also allowing EKS to deploy 1000s of pods and minimizing the impact on the corporate CIDR. So, yeah, the primary VPC CIDR is only /24 (routed), and the secondary VPC CIDR is /16 (not routed). Before Edit: Tried deploying the node group to the secondary subnets, but that didn't work without routing. I can get it working using a Private NAT Gateway. Not sure that's worth it. Sounding like back to Custom Networking with ENIConfig CRDs :( |
I'll go ahead and close this. The combination of |
This issue is now closed. Comments on closed issues are hard for our team to see. |
What happened:
Setting
ENABLE_PREFIX_DELEGATION=true
andENABLE_SUBNET_DISCOVERY=true
does not appear to allocate IPs from secondary subnets tagged withkubernetes.io/role/cni=1
. I'm pretty new to K8S and EKS, so rather likely I'm doing something wrong. But if I changeENABLE_PREFIX_DELEGATION
tofalse
, then the IPs are allocated from the secondary subnets.To get prefix delegation working with the secondary subnets, I can set things up using
AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true
ENI_CONFIG_LABEL_DEF=topology.kubernetes.io/zone
and an ENIConfig custom resource for each availability zone. But I was hoping to simplify things by just tagging the subnets withkubernetes.io/role/cni=1
and removing the custom network config.Environment:
kubectl version
):v1.30.0-eks-036c24b
v1.18.2-eksbuild.1
cat /etc/os-release
):Amazon Linux 2023.4.20240528
uname -a
):Linux ip-10-200-2-107 6.1.91-99.172.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 21 16:55:08 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
The text was updated successfully, but these errors were encountered: