Terraform destroy being stuck

[keyvaann]

It's Known limitation mentioned in the README that terraform destroy won't work unless helmfile destroy has been ran before. I don't think this is very practical and it's better if we find a solution for it.

I searched the issue and it seems like there is a solution for it and the launch template of the node groups should be changed to auto delete the resources that are created by the EKS.

I'll update the issue when more information later.

[keyvaann]

The issue is discussed in here aws/amazon-vpc-cni-k8s#69
It seems like with enabling delete_on_termination flag, the issue will be resolved.

[keyvaann]

It seems that configuration flag isn't available anymore. An alternative option would be to use delete time provisioners to delete all services on the cluster before its removal, but I don't think that would be a best practice
https://developer.hashicorp.com/terraform/language/resources/provisioners/syntax#destroy-time-provisioners

Or maybe creating the ingress controller with the terraform so it will also remove those before removing the cluster.

[baixiac]

That means the ingress controller needs to be created after the infrastructure is ready, which seems to go against your proposal of merging cluster and config folders into one.

[keyvaann]

I think we can have merging both folders and releasing an ingress controller via terraform.
In this case probably using ALB ingress controller would make more sense since it is required if you want to use AWS WAF.

[baixiac]

I remember there are a few reasons why NLB is a better choice than ALB:
https://aws.amazon.com/blogs/opensource/network-load-balancer-nginx-ingress-controller-eks/#bGA9CArTgFU
It is also possible to put ALB in front of NLB if WAF has to be used.

[keyvaann]

In our current setup with a ALB sit on top of the default Nginx controller that is installed via RADAR-Kubernetes, which isn't ideal but I think it mitigates the issue mentioned in the shared link.
I'm not sure what is the best way to solve this issue at the moment.