Support scoped serviceAccount

[lahabana]

Description

• Prior to 1.30.6, When OIDC issued service token with --service-account-extend-token-expiration set to true . With this, the service account tokens injected into pods were given an extended lifetime so they remain valid even after a new refreshed token is provided. Prior to upgrading to 1.30, the metric serviceaccount_stale_tokens_total and the audit annotation authentication.k8s.io/stale-token can be used to monitor for workloads that depend on the extended lifetime and are continuing to use tokens even after a refreshed token is provided to the container. If no action is taken, workloads depending on the extended lifetime will break once the cluster is upgraded to 1.30. See reference README.md for details.

After upgrading to 1.30.6 , It is no longer supported to change “service-account-extend-token-expiration” setting back to “True” for 1.30 OIDC enabled cluster thus this is the reason Kong Mesh CNI and egress pod were not able to renew the token and broke down with Unathorized.

We should support bounded serviceAccount as it seems to be an important security new feature in 1.30.

Docs:

https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#manually-create-a-long-lived-api-token-for-a-serviceaccount
https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md

Is this only for CNI or more generally for other components?

[jijiechen]

Correction: the bound service account feature was introduced in Kubernetes 1.22. They allow the switch service-account-extend-token-expiration to play a "deprecation period" support duing v1.22 to v1.30 and this switch was eventually removed in 1.30.

Fortunately, from my study of Kuma code base, only Kuma-CNI is being impacted by this issue.

In other components, we are either not interacting with the API server directly or not copying the token to somewhere else before we use it. The client-go will read the token file from disk every minute (link to source) and the content of the token file is maintained by Kubernetes automatically. So this is NOT an issue for components using the offical Go SDK without storing somewhere else or caching the token.

Note: If having trouble in finding places using in-cluster config completely, cluster operators can specify flag --service-account-extend-token-expiration=true to kube apiserver to allow tokens have longer expiration temporarily during the migration. Any usage of legacy token will be recorded in both metrics and audit logs. After fixing all the potentially broken workloads, turn off the flag so that the original expiration settings are honored. Note the --service-account-extend-token-expiration mitigation defaults to true, and that cluster administrators can set it to --service-account-extend-token-expiration=false to turn off the mitigation if desired.

Source: https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md#serviceaccount-admission-controller-migration

[michaelbeaumont]

I think #11399 solved this for kuma-dp

[jijiechen]

We are using the token to authenticate DPs and this is not fixed in older versions. So our DPs, including the ingress and egress are also impacted

[bartsmykla]

We are using the token to authenticate DPs and this is not fixed in older versions. So our DPs, including the ingress and egress are also impacted

Backporting done:

• fix(k8s): always authenticate with latest service account token (backport #11399) #12586
• fix(k8s): always authenticate with latest service account token (backport #11399) #12587
• fix(k8s): always authenticate with latest service account token (backport #11399) #12588