Authentication fails for Keda with AWS IRSA role

[harshit2202]

Hi,

I have an app deployed on k8s, let's say it app-1. It has an IRSA role attached to its service account and the policy is

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "sqs:*",
            "Effect": "Allow",
            "Resource": "*",
            "Sid": ""
        }
    ]
}

and the trust definition is

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::demo-account:oidc-provider/oidc.eks.us-east-2.amazonaws.com/id/demo-id"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.us-east-2.amazonaws.com/id/demo-id:sub": "system:serviceaccount:keda:keda-operator",
                    "oidc.eks.us-east-2.amazonaws.com/id/demo-id:aud": "sts.amazonaws.com"
                }
            }
        }
    ]
}

After this, I installed the keda on k8s using helm
helm install keda kedacore/keda --namespace keda

Post the installation, I created a Triggered Authentication and ScaledObject as below

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: {{ $trigger.name}}
  namespace: {{ .Namespace }}
spec:
  podIdentity:
    provider: aws
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: {{ .Name }}
  annotations:
    scaledobject.keda.sh/transfer-hpa-ownership: "true"
    ...
spec:
  scaleTargetRef:
    apiVersion: apps/v1                                 
    kind: {{ .targetRefType }}               
    name: {{ .targetRefName }}              
  .....
  advanced:
    horizontalPodAutoscalerConfig:
      name: {{ .hpaName }}
  triggers:
    - type: {{ $trigger.type }}
      authenticationRef:
        name: {{ $trigger.name }}
      name: "{{ .Name }}"
      metadata:
        ...
      useCachedMetrics: {{ .UseCachedMetrics }}

The keda-operator logs are still saying 403 Access denied. The exact log message is

2024-06-06T13:18:42Z    ERROR    scale_handler    error getting metric for trigger    {"scaledObject.Namespace": "app-1", "scaledObject.Name": "app-1", "trigger":  "app-1-aws-sqs-queue", "error": "operation error SQS: GetQueueAttributes, get identity: get credentials: failed to refresh cached credentials, operation error STS: AssumeRole, https response error StatusCode: 403, RequestID: 9a55f153-9570-4754-9930-52222401255a, api error AccessDenied: User: arn:aws:sts::demo-account:assumed-role/demo-eks/i-0ed3123456f123456 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::demo-account:role/app-1"}    

Can anyone help for this? TIA :)