Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Work around the limit in lambda policy size #61

Merged
merged 1 commit into from
Sep 30, 2024
Merged

Conversation

zbstof
Copy link
Contributor

@zbstof zbstof commented Sep 30, 2024

Currently failing with

│ Error: adding Lambda Permission (arn:aws:lambda:us-east-2:591234544403:function:model-inference-development-datadog-forwarder/_aws_sagemaker_Endpoints_recs-ranker-rn-scribd-has-read-after-AllowExecutionFromCloudWatchLogs): PolicyLengthExceededException: The final policy size (20782) is bigger than the limit (20480).
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "e220c7fb-c4c9-4b93-9f15-beba592a3afc"
│   },
│   Message_: "The final policy size (20782) is bigger than the limit (20480).",
│   Type: "User"
│ }
│
│   with module.datadog[0].aws_lambda_permission.allow_cloudwatch_logs_to_call_dd_lambda_handler["/aws/sagemaker/Endpoints/recs-ranker-rn-scribd-has-read-after"],
│   on .terraform/modules/datadog/logs_monitoring_cloudwatch_log.tf line 10, in resource "aws_lambda_permission" "allow_cloudwatch_logs_to_call_dd_lambda_handler":
│   10: resource "aws_lambda_permission" "allow_cloudwatch_logs_to_call_dd_lambda_handler" {
}

Current size of lambda policy:

$ aws lambda get-policy --function-name model-inference-development-datadog-forwarder | jq -r '.Policy' | wc
       1       1   20090

Now we can pass array like log_group_prefixes = ["/aws/sagemaker/Endpoints/", "/aws/sagemaker/TransformJobs"] to force small size of the policy if necessary

Changes to the module are backward-compatible

Currently failing with
```
│ Error: adding Lambda Permission (arn:aws:lambda:us-east-2:591234544403:function:model-inference-development-datadog-forwarder/_aws_sagemaker_Endpoints_recs-ranker-rn-scribd-has-read-after-AllowExecutionFromCloudWatchLogs): PolicyLengthExceededException: The final policy size (20782) is bigger than the limit (20480).
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "e220c7fb-c4c9-4b93-9f15-beba592a3afc"
│   },
│   Message_: "The final policy size (20782) is bigger than the limit (20480).",
│   Type: "User"
│ }
│
│   with module.datadog[0].aws_lambda_permission.allow_cloudwatch_logs_to_call_dd_lambda_handler["/aws/sagemaker/Endpoints/recs-ranker-rn-scribd-has-read-after"],
│   on .terraform/modules/datadog/logs_monitoring_cloudwatch_log.tf line 10, in resource "aws_lambda_permission" "allow_cloudwatch_logs_to_call_dd_lambda_handler":
│   10: resource "aws_lambda_permission" "allow_cloudwatch_logs_to_call_dd_lambda_handler" {
}
```

Current size of lambda policy:
```
$ aws lambda get-policy --function-name model-inference-development-datadog-forwarder | jq -r '.Policy' | wc
       1       1   20090
```

Now we can pass array like `log_group_prefixes = ["/aws/sagemaker/Endpoints/", "/aws/sagemaker/TransformJobs"]`
to force small size of the policy if necessary

Changed to the module are backward-compatible
@zbstof zbstof changed the title fix: Work around limit in lambda policy size fix: Work around the limit in lambda policy size Sep 30, 2024
@zbstof zbstof merged commit f970130 into master Sep 30, 2024
4 checks passed
@zbstof zbstof deleted the fix-policy-size branch September 30, 2024 13:07
Copy link

🎉 This PR is included in version 3.3.2 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

1 participant