From 69cd940d2d7fb60cee5924161da27bb45fd656c8 Mon Sep 17 00:00:00 2001 From: Joeri Malmberg Date: Thu, 22 Jun 2023 16:08:50 +0200 Subject: [PATCH 1/2] Updated README and removed a few redundant configuration options. --- README.md | 65 ++++++++++--------- .../management/00-organization/terragrunt.hcl | 2 +- cloud/management/terragrunt.hcl | 4 +- global.hcl | 24 ++++--- 4 files changed, 55 insertions(+), 40 deletions(-) diff --git a/README.md b/README.md index 129ccbd..e07571f 100644 --- a/README.md +++ b/README.md @@ -1,52 +1,59 @@ # Blackbird Cloud AWS Cloud Environment Template -[![blackbird-logo](https://raw.githubusercontent.com/blackbird-cloud/terraform-module-template/main/.config/logo_simple.png)](https://blackbird.cloud) +[![blackbird-logo](https://raw.githubusercontent.com/blackbird-cloud/terraform-module-template/main/.config/logo_simple.png)](https://www.blackbird.cloud) ## Intro We (Blackbird Cloud) have deployed many AWS cloud environment for our clients. We use this repository as a boilerplate for our cloud deployment. This Repository includes: -* Account management -* Organization management -* GitHub OIDC Stack +* AWS Cloudformation Stack templates to bootstrap your account after creation. +* Terragrunt and Terraform resources to configure the following services: + * AWS Organizations + * AWS IAM Identity Center + * AWS Cloudformation StackSets + * AWS S3 bucket for Terraform state storage. + * AWS KMS key for Terraform state encryption. * GitOps (GitHub Action) pipeline -* Stacksets for common resources in every accounts (TF State Bucket, GitHub Role, etc) ## How to deploy -1. Create AWS Account, make sure you note down the account name and account id -2. Go to `cloud/00-organization/terragrunt.hcl` and fill in your company detail -3. Deploy Both `stacks/github-oidc-provider.yaml` and `stacks/terraform-state.yaml` to CloudFormation - * For github oidc provider stack fill in `SubjectClaimFilters` with the following format `repo:YOUR_ORGANIZATION/YOUR_REPO:ref:refs/heads/BRANCH_NAME` we advise to deploy use `main` as branch name. This is nessecary to make sure that only GitHub Actions that run on the main branch are allowed to plan and apply changes on AWS. - * Once the terraform state stack is done note down the bucket name, it will be used as the state bucket for the next steps. -4. Create 2 variables on GitHub -> Settings -> Secrets and variables -> Actions -> Variables +1. Create an AWS Account [here](https://portal.aws.amazon.com/billing/signup#/start/email]), name it management. Select the region you would like to deploy your resources to, write down the region and account id. +2. Navigate to AWS Cloudformation => Stacks, and manually deploy both `stacks/github-oidc-provider.yaml` and `stacks/terraform-state.yaml`. + * For the GitHub oidc provider stack, fill in `SubjectClaimFilters` with the following data relating to your infra repo `repo:YOUR_GITHUB_ORGANIZATION/YOUR_INFRA_REPO:ref:refs/heads/BRANCH_NAME` we advise to deploy use `main` as branch name. This is nessecary to make sure that only GitHub Actions that run on the main branch are allowed to plan and apply changes on AWS. Make sure to protect your main branch, as it will receive AdministratorAccess on your AWS cloud. Once the stack has been created, navigate to its resources, and note down the arn of created IAM role. + * Once the terraform state stack has been created, note down the bucket name, it will be used as the state bucket for the next steps. +3. Create 2 variables on GitHub -> Settings -> Secrets and variables -> Actions -> Variables * `AWS_IAM_ROLE`: fill in `IAM Role ARN` created by github-oidc-provider stack - * `AWS_REGION`: fill in the region where you created the CloudFormation Stacks on step 3 -5. On `.github/workflows/aws_deployment.yml` update all occurences of `my-project-name` to your github project name. -6. On `cloud/global.hcl` enter all the information under `MANUAL STEP: Enter manually` block -7. On` cloud/management/terragrunt.hcl` enter all the information under `MANUAL STEP: Enter manually` block. Use the bucket name created by `stacks/terraform-state.yaml` for `bucket_name` -8. Create the users list on `cloud/management/02-iam-sso/01-users/terragrunt.hcl`, you can remove `john.doe@email.com`. -9. On `cloud/management/02-iam-sso/02-groups/terragrunt.hcl` enter the groups with the users you would like to create. Make sure to assign the users created by adding multiple `dependency.users.outputs.users["USER_EMAIL"].user_id` and replace `USER_EMAIL` with the actual email. + * `AWS_REGION`: fill in your selected AWS region. +4. On `.github/workflows/aws_deployment.yml` update all occurences of `` to your github repository name, line 37. + +5. On `cloud/global.hcl` enter all the required information at the `Enter manually` block. +6. On` cloud/management/terragrunt.hcl` enter all the information under `Enter manually` block. Use the bucket name created by `stacks/terraform-state.yaml` for `bucket_name`, and enter the account id from the AWS account you created in step 1. +7. Go to `cloud/00-organization/terragrunt.hcl` and fill in the primary, operational, securit, and billing contact information. + +8. (Optional) If your IDP supports provisioning users and groups, you can skip this step, and delete the `cloud/management/02-iam-sso/01-users` folder, and the `cloud/management/02-iam-sso/02-groups` folder. + * Create the users list on `cloud/management/02-iam-sso/01-users/terragrunt.hcl`, you can remove `john.doe@email.com`. + * `cloud/management/02-iam-sso/02-groups/terragrunt.hcl` enter the groups with the users you would like to create. Make sure to assign the users created by adding multiple `dependency.users.outputs.users["USER_EMAIL"].user_id` and replace `USER_EMAIL` with the actual email. * On initial run align the `mock_output` value of `dependency "users"` with `01-users`, make sure all emails registered `01-users` are `listed`. `user_id` value can be left `"user_id"` -10. (Optional) On `cloud/management/02-iam-sso/03-permission-sets/terragrunt.hcl` enter the permission-sets you would like to create. We've included our default permission sets that we use for common use cases -11. (Optional) On `cloud/management/02-iam-sso/04-account-assignment/terragrunt.hcl` assign accounts and permission-sets, to users and groups. The default value will deploy `AdministratorAccess` permission set along with attached users on all accounts. -12. Commit and push, it will trigger the pipeline to run + +10. (Optional) On `cloud/management/02-iam-sso/03-permission-sets/terragrunt.hcl` enter the permission-sets you would like to create. We have included some commonly used permission-sets. +11. (Optional) On `cloud/management/02-iam-sso/04-account-assignment/terragrunt.hcl` assign accounts and permission-sets, to users and groups. The default value will deploy the `AdministratorAccess` permission set for the Administrators group. +12. Commit and push, it will trigger the pipeline to run. * It will *fail* initially 13. Then there are a few steps to be taken before re-runing the pipeline - * Go the management account, navigate to stackset then enable trusted access - * Go to IAM identity center, and click on enable then Configure your IdP -14. Re-run the pipeline, and fingers crossed everything should run + * Open your AWS web console and navigate to Cloudformation => StackSets, then enable trusted access. + * Open your AWS web console and navigate to IAM Identity Center, then click on enable. + * Open your AWS web console and navigate to IAM Identity Center => Settings. At the identity source tab, click Actions and select change identity source. Read the documentation [here](https://docs.aws.amazon.com/singlesignon/latest/userguide/manage-your-identity-source.html) to proceed depending your Organization's IDP. +14. Re-run the pipeline and all IAM and StackSets should now deploy succesfully. -## Troubleshoot +## Troubleshooting ### Rate Limited ``` -Error: enabling Security Hub Organization Admin Account (741117833771): LimitExceededException: AWS Organizations can't complete your request because another request is already in progress. Try again later. +Error: enabling Security Hub Organization Admin Account (XXXXXXXXX): LimitExceededException: AWS Organizations can't complete your request because another request is already in progress. Try again later. ``` -If you see this error, it means we're beeing rate limited by AWS. Simply re-run the action to fix it - +If you see this error, it means you are being rate limited by AWS. Simply re-run the failed pipeline and give it another shot. ## Future improvements @@ -58,10 +65,10 @@ If you see this error, it means we're beeing rate limited by AWS. Simply re-run ## About Blackbird Cloud -We are [Blackbird Cloud](https://blackbird.cloud), Amsterdam based cloud consultancy, and cloud management service provider. We help companies build secure, cost efficient, and scale-able solutions. +We are [Blackbird Cloud](https://www.blackbird.cloud), Amsterdam based cloud consultancy, and cloud management service provider. We help companies build secure, cost efficient, and scale-able solutions. Checkout our other :point_right: [terraform modules](https://registry.terraform.io/namespaces/blackbird-cloud) ## Copyright -Copyright © 2017-2023 [Blackbird Cloud](https://blackbird.cloud) \ No newline at end of file +Copyright © 2017-2023 [Blackbird Cloud](https://www.blackbird.cloud) \ No newline at end of file diff --git a/cloud/management/00-organization/terragrunt.hcl b/cloud/management/00-organization/terragrunt.hcl index 3cd0f25..ae9b673 100644 --- a/cloud/management/00-organization/terragrunt.hcl +++ b/cloud/management/00-organization/terragrunt.hcl @@ -67,7 +67,7 @@ inputs = { accounts = [ { name = "security-tooling" - email = "info+${local.global.account_email_label_prefix}-security-tooling@${local.global.account_email_domain}" + email = "info+${local.global.repository}-security-tooling@${local.global.account_email_domain}" delegated_administrator_services = [ "config.amazonaws.com", "guardduty.amazonaws.com", diff --git a/cloud/management/terragrunt.hcl b/cloud/management/terragrunt.hcl index b1a2e97..aaeb335 100644 --- a/cloud/management/terragrunt.hcl +++ b/cloud/management/terragrunt.hcl @@ -1,5 +1,5 @@ locals { - ## MANUAL STEP: Enter manually + ## Enter manually account_id = "" bucket = "" ## =========================== @@ -36,7 +36,7 @@ remote_state { inputs = { tags = { terragrunt = get_path_from_repo_root() - project = local.global.project + repository = local.global.repository } } diff --git a/global.hcl b/global.hcl index d9ec2aa..40e2b8a 100644 --- a/global.hcl +++ b/global.hcl @@ -1,13 +1,21 @@ locals { - account_email_label_prefix = local.project - administrator_role_name = "AdministratorAccess" - github_role_name = "GitHub" + repository = split("/", get_repo_root())[0] + github_role_name = "GitHub" + # For local usage, configure your credentialls under a profile with the following naming convention: + # --${local.administrator_role_name} + # This defaults to AdministratorAccess, but if you use something else, please update it below. + administrator_role_name = "AdministratorAccess" + # The name of your company. Used to for local AWS access, must match with your profile naming convention. + organization_name = "myorg" - ### MANUAL STEP: Enter manually - region = "eu-central-1" - organization_name = "myorg" - project = "mycloud" - github_role_arn = "" + ### Enter manually + # The selected AWS region to create all cloud resources. + region = "eu-central-1" + + # The ARN of the role created by the github-oidc-provider Cloudformation template. + github_role_arn = "" + + # The email domain used for creating AWS accounts by the Organization module account_email_domain = "acme.com" ## =========================== } From 8f4dddf6332f31883af39c60d9e0797396d57de6 Mon Sep 17 00:00:00 2001 From: Joeri Malmberg Date: Thu, 22 Jun 2023 16:10:12 +0200 Subject: [PATCH 2/2] Small tweak --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e07571f..150165c 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ This Repository includes: 1. Create an AWS Account [here](https://portal.aws.amazon.com/billing/signup#/start/email]), name it management. Select the region you would like to deploy your resources to, write down the region and account id. 2. Navigate to AWS Cloudformation => Stacks, and manually deploy both `stacks/github-oidc-provider.yaml` and `stacks/terraform-state.yaml`. - * For the GitHub oidc provider stack, fill in `SubjectClaimFilters` with the following data relating to your infra repo `repo:YOUR_GITHUB_ORGANIZATION/YOUR_INFRA_REPO:ref:refs/heads/BRANCH_NAME` we advise to deploy use `main` as branch name. This is nessecary to make sure that only GitHub Actions that run on the main branch are allowed to plan and apply changes on AWS. Make sure to protect your main branch, as it will receive AdministratorAccess on your AWS cloud. Once the stack has been created, navigate to its resources, and note down the arn of created IAM role. + * For the GitHub oidc provider stack, fill in `SubjectClaimFilters` with the following data relating to your infra repo `repo:YOUR_GITHUB_ORGANIZATION/YOUR_GITHUB_REPOSITORY_NAME:ref:refs/heads/BRANCH_NAME` we advise to deploy use `main` as branch name. This is nessecary to make sure that only GitHub Actions that run on the main branch are allowed to plan and apply changes on AWS. Make sure to protect your main branch, as it will receive AdministratorAccess on your AWS cloud. Once the stack has been created, navigate to its resources, and note down the arn of created IAM role. * Once the terraform state stack has been created, note down the bucket name, it will be used as the state bucket for the next steps. 3. Create 2 variables on GitHub -> Settings -> Secrets and variables -> Actions -> Variables * `AWS_IAM_ROLE`: fill in `IAM Role ARN` created by github-oidc-provider stack