[efs-controller] - reconciler error when enable the backup

[Yunbo-Lu]

Describe the bug
When I deployed the FileSystem manifest with the spec.backup field set to true, I saw the EFS was created correctly with backup setting enabled in AWS console, but I saw the following error message in the EFS controller log, which complained aws:elasticfilesystem:default-backup tag is invalid.

{"level":"error","ts":"2024-09-30T18:11:12.252Z","msg":"Reconciler error","controller":"filesystem","controllerGroup":"efs.services.k8s.aws","controllerKind":"FileSystem","FileSystem":{"name":"my-filesystem","namespace":"ack-system"},"namespace":"ack-system","name":"my-filesystem","reconcileID":"5c4a794e-775d-4022-b2e0-63ca2aef5307","error":"ValidationException: 1 validation error detected: Value '[aws:elasticfilesystem:default-backup]' at 'tagKeys' failed to satisfy constraint: Member must satisfy constraint: [Member must have length less than or equal to 128, Member must have length greater than or equal to 1, Member must satisfy regular expression pattern: ^(?![aA]{1}[wW]{1}[sS]{1}:)([\\p{L}\\p{Z}\\p{N}_.:/=+\\-@]+)$]\n{\n RespMetadata: {\n StatusCode: 400,\n RequestID: \"e6f996e4-5553-4734-ac84-07cf979e1389\"\n },\n Message_: \"1 validation error detected: Value '[aws:elasticfilesystem:default-backup]' at 'tagKeys' failed to satisfy constraint: Member must satisfy constraint: [Member must have length less than or equal to 128, Member must have length greater than or equal to 1, Member must satisfy regular expression pattern: ^(?![aA]{1}[wW]{1}[sS]{1}:)([\\\\p{L}\\\\p{Z}\\\\p{N}_.:/=+\\\\-@]+)$]\"\n}","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}

Then, when I check the the filesystem's status in EKS, the SYNCED field is shown as Unknown as a result:

➜ kubectl get filesystem -n ack-system  
NAME                ID                                           ENCRYPTED   SIZE   MOUNTTARGETS   STATE       SYNCED    AGE
my-filesystem   fs-00000000000000000   true                 6144   0                              available   Unknown   73s

Steps to reproduce
Deploy FileSytem manifest with backup set to true

Expected outcome
A concise description of what you expected to happen.
There will be no error in EFS controller log when deploying FileSytem manfiest with backup enabled and the SYNCED status should be true after the EFS is created

Environment

• Kubernetes version 1.29
• Using EKS (yes/no), if so version? Yes, eks.12
• AWS service targeted (S3, RDS, etc.) EFS