Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added new cwli query to track sender IDs for international SMS #1616

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Added new cwli query to track sender IDs for international SMS #1616

wants to merge 2 commits into from

Conversation

jimleroyer
Copy link
Member

Summary | Résumé

  • Track sender IDs that are used for SMS sent at the international, via a CloudWatch query.

Related Issues | Cartes liées

  • None

Before merging this PR

Read code suggestions left by the
cds-ai-codereviewer bot. Address
valid suggestions and shortly write down reasons to not address others. To help
with the classification of the comments, please use these reactions on each of the
comments made by the AI review:

Classification Reaction Emoticon
Useful +1 👍
Noisy eyes 👀
Hallucination confused 😕
Wrong but teachable rocket 🚀
Wrong and incorrect -1 👎

The classifications will be extracted and summarized into an analysis of how helpful
or not the AI code review really is.

Test instructions | Instructions pour tester la modification

You can test the query in production as it was saved via clickops in that env.

Release Instructions | Instructions pour le déploiement

None.

Reviewer checklist | Liste de vérification du réviseur

  • This PR does not break existing functionality.
  • This PR does not violate GCNotify's privacy policies.
  • This PR does not raise new security concerns. Refer to our GC Notify Risk Register document on our Google drive.
  • This PR does not significantly alter performance.
  • Additional required documentation resulting of these changes is covered (such as the README, setup instructions, a related ADR or the technical documentation).

⚠ If boxes cannot be checked off before merging the PR, they should be moved to the "Release Instructions" section with appropriate steps required to verify before release. For example, changes to celery code may require tests on staging to verify that performance has not been affected.

@jimleroyer jimleroyer self-assigned this Oct 22, 2024
@github-actions GitHub Actions
Copy link

staging: common

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

⚠️   Warning: resources will be destroyed by this change!

Plan: 4 to add, 0 to change, 4 to destroy
Show summary
CHANGE NAME
recreate module.notify_slack_critical.module.lambda.null_resource.archive[0]
module.notify_slack_general.module.lambda.null_resource.archive[0]
module.notify_slack_ok.module.lambda.null_resource.archive[0]
module.notify_slack_warning.module.lambda.null_resource.archive[0]
Show plan 
Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.notify_slack_critical.module.lambda.null_resource.archive[0] must be replaced
-/+ resource "null_resource" "archive" {
      ~ id       = "1896303228432941431" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "timestamp" = "1729629187439733000" -> "1729637386145390000"
            # (1 unchanged element hidden)
        }
    }

  # module.notify_slack_general.module.lambda.null_resource.archive[0] must be replaced
-/+ resource "null_resource" "archive" {
      ~ id       = "9024575385196300947" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "timestamp" = "1729629178574569000" -> "1729637379237118000"
            # (1 unchanged element hidden)
        }
    }

  # module.notify_slack_ok.module.lambda.null_resource.archive[0] must be replaced
-/+ resource "null_resource" "archive" {
      ~ id       = "8575330831001695155" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "timestamp" = "1729629187442029000" -> "1729637386142119800"
            # (1 unchanged element hidden)
        }
    }

  # module.notify_slack_warning.module.lambda.null_resource.archive[0] must be replaced
-/+ resource "null_resource" "archive" {
      ~ id       = "7066818209283720979" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "timestamp" = "1729629187430103000" -> "1729637386129942000"
            # (1 unchanged element hidden)
        }
    }

Plan: 4 to add, 0 to change, 4 to destroy.

Warning: Argument is deprecated

  with aws_s3_bucket.csv_bucket,
  on s3.tf line 5, in resource "aws_s3_bucket" "csv_bucket":
   5: resource "aws_s3_bucket" "csv_bucket" {

Use the aws_s3_bucket_server_side_encryption_configuration resource instead

(and 65 more similar warnings elsewhere)

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results 
WARN - plan.json - main - Missing Common Tags: ["aws_athena_workgroup.ad_hoc"]
WARN - plan.json - main - Missing Common Tags: ["aws_athena_workgroup.build_tables"]
WARN - plan.json - main - Missing Common Tags: ["aws_athena_workgroup.primary"]
WARN - plan.json - main - Missing Common Tags: ["aws_athena_workgroup.support"]
WARN - plan.json - main - Missing Common Tags: ["aws_budgets_budget.notify_global"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_event_rule.aws_health[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.route53_resolver_query_log[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.sns_deliveries[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.sns_deliveries_failures[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.sns_deliveries_failures_us_west_2[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.sns_deliveries_us_west_2[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-bulk-not-being-processed-critical[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-bulk-not-being-processed-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-inflights-not-being-processed-critical[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-inflights-not-being-processed-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-not-being-processed-critical[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-not-being-processed-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.contact-3-500-error-15-minutes-critical[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.document-download-bucket-size-warning[0]"]
WARN - plan.json...

@github-actions GitHub Actions
Copy link

staging: quicksight

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

Plan: 0 to add, 1 to change, 0 to destroy
Show summary
CHANGE NAME
update aws_s3_object.manifest_file
Show plan 
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_s3_object.manifest_file will be updated in-place
  ~ resource "aws_s3_object" "manifest_file" {
      ~ etag                          = "4f558e8d8cdbbf914a95755cbda61968" -> "221f592f333f2fc284626cfdb8c4bc80"
        id                            = "quicksight/s3-manifest-sms-usage.json"
        tags                          = {}
      + version_id                    = (known after apply)
        # (24 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results 
WARN - plan.json - main - Missing Common Tags: ["aws_cloudformation_stack.sms-usage-notifications"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight-rds"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight-s3-usage"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight_vpc_connection_ec2"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight_vpc_connection_iam"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_role.quicksight"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_role.vpc_connection_role"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.jobs"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.login_events"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.notifications"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.organisation"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.services"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.sms_usage"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.template-category-history"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.templates"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.users"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_source.rds"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_source.s3_sms_usage"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_vpc_connection.rds"]
WARN - plan.json - main - Missing Common Tags: ["aws_s3_object.manifest_file"]

39 tests, 19 passed, 20 warnings, 0 failures, 0 exceptions

sastels
sastels reviewed Oct 25, 2024
View reviewed changes
aws/pinpoint_to_sqs_sms_callbacks/cloudwatch_queries.tf

log_group_names = [
aws_cloudwatch_log_group.pinpoint_deliveries.name,
aws_cloudwatch_log_group.pinpoint_deliveries_failures.name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way we have sms set up (sms and pinpoint), the deliveries log groups gets everything while the deliveries_failures groups just get the failures, so I think here we just want the deliveries log group or else failures will be duplicated.

@github-actions GitHub Actions
Copy link

staging: pinpoint_to_sqs_sms_callbacks

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

Plan: 1 to add, 0 to change, 0 to destroy
Show summary
CHANGE NAME
add aws_cloudwatch_query_definition.pinpoint-sms-international-sending-status-sender-id[0]
Show plan 
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_cloudwatch_query_definition.pinpoint-sms-international-sending-status-sender-id[0] will be created
  + resource "aws_cloudwatch_query_definition" "pinpoint-sms-international-sending-status-sender-id" {
      + id                  = (known after apply)
      + log_group_names     = [
          + (sensitive value),
          + (sensitive value),
        ]
      + name                = "SMS (Pinpoint) / International sending status (with Sender IDs)"
      + query_definition_id = (known after apply)
      + query_string        = <<-EOT
            fields @timestamp, @message
            | parse @message /"isoCountryCode":"(?<Country>[^"]+)"/
            | parse @message /"eventType":"(?<EventType>[^"]+)"/
            | parse @message /"isFinal":(?<Is_Final>\w+)/
            | filter Is_Final = "true"
            | parse originationPhoneNumber /(?<longCode>\+\d{11})/
            | parse originationPhoneNumber /(?<senderId>^[^\+][a-zA-Z]+)/
            | parse originationPhoneNumber /(?<shortCode>237762)/
            | stats
                count(longCode) as TotalLongCode,
                count(shortCode) as TotalShortCode,
                count(senderId) as TotalSenderId
                by Country, EventType, senderId
            | sort Country asc, EventType asc
        EOT
    }

Plan: 1 to add, 0 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results 
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.pinpoint_deliveries"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.pinpoint_deliveries_failures"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.pinpoint_to_sqs_sms_callbacks_log_group[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.lambda-image-pinpoint-delivery-receipts-errors-critical[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.lambda-image-pinpoint-delivery-receipts-errors-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.logs-1-500-error-1-minute-warning-pinpoint_to_sqs_sms_callbacks-api[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.logs-10-500-error-5-minutes-critical-pinpoint_to_sqs_sms_callbacks-api[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.pinpoint-sms-blocked-as-spam-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.pinpoint-sms-failures-bell-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.pinpoint-sms-failures-bragg-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.pinpoint-sms-failures-freedom-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.pinpoint-sms-failures-iristel-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.pinpoint-sms-failures-maritime-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.pinpoint-sms-failures-mts-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.pinpoint-sms-failures-rogers-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.pinpoint-sms-failures-telus-warning[0]"]
WARN - plan.json - main - Missing...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sastels sastels sastels left review comments

@ben851 ben851 Awaiting requested review from ben851

@P0NDER0SA P0NDER0SA Awaiting requested review from P0NDER0SA

At least 1 approving review is required to merge this pull request.

Assignees

@jimleroyer jimleroyer

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jimleroyer @sastels