From 53ca99fee1175d665ec6bbc71d4069eafecaa97d Mon Sep 17 00:00:00 2001 From: Anton Babenko Date: Thu, 29 Aug 2019 10:30:39 +0200 Subject: [PATCH] Rewrite to match other modules, added all existing S3 features --- .pre-commit-config.yaml | 4 +- README.md | 122 +++++++----------- examples/complete/README.md | 40 ++++++ examples/complete/main.tf | 133 ++++++++++++++++++++ examples/complete/outputs.tf | 39 ++++++ examples/s3-cors/main.tf | 34 ------ examples/s3-lifecycle-rule/main.tf | 67 ---------- examples/s3-logging/main.tf | 32 ----- examples/s3-replication/README.md | 33 +++++ examples/s3-replication/iam.tf | 65 ++++++++++ examples/s3-replication/main.tf | 145 ++++++++++++++++------ examples/s3-replication/outputs.tf | 39 ++++++ examples/s3-versioning/main.tf | 25 ---- examples/s3-website/main.tf | 36 ------ main.tf | 190 ++++++++++++++++------------- outputs.tf | 16 +-- variables.tf | 182 +++++++++------------------ 17 files changed, 678 insertions(+), 524 deletions(-) create mode 100644 examples/complete/README.md create mode 100644 examples/complete/main.tf create mode 100644 examples/complete/outputs.tf delete mode 100644 examples/s3-cors/main.tf delete mode 100644 examples/s3-lifecycle-rule/main.tf delete mode 100644 examples/s3-logging/main.tf create mode 100644 examples/s3-replication/README.md create mode 100644 examples/s3-replication/iam.tf create mode 100644 examples/s3-replication/outputs.tf delete mode 100644 examples/s3-versioning/main.tf delete mode 100644 examples/s3-website/main.tf diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 150a8c85..6685b961 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,10 +1,10 @@ repos: - repo: git://github.com/antonbabenko/pre-commit-terraform - rev: v1.17.0 + rev: v1.19.0 hooks: - id: terraform_fmt - id: terraform_docs - repo: git://github.com/pre-commit/pre-commit-hooks - rev: v2.2.3 + rev: v2.3.0 hooks: - id: check-merge-conflict diff --git a/README.md b/README.md index 37e64d0d..25aeb091 100644 --- a/README.md +++ b/README.md @@ -1,29 +1,21 @@ # AWS S3 bucket Terraform module -Terraform module which creates S3 bucket resources on AWS. +Terraform module which creates S3 bucket on AWS with all (or almost all) features provided by Terraform AWS provider. This type of resources are supported: * [S3 bucket](https://www.terraform.io/docs/providers/aws/r/s3_bucket.html) -These S3 Bucket configurations are supported: +These features of S3 bucket configurations are supported: -- cors -- lifecycle-rules -- logging -- replication (Cross Region Replication - CRR)* +- static web-site hosting +- access logging - versioning -- website - -``` -These configurations are not supported yet: - -In Cross Region Replication (in replication_configuration/rules block): -- priority (the argument is not supported yet). -- filter (the argument is not supported yet). - -Object Lock Configuration block(object_lock_configuration) (this configuration block is not supported yet). -``` +- CORS +- lifecycle rules +- server-side encryption +- object locking +- Cross-Region Replication (CRR) ## Terraform versions @@ -31,62 +23,41 @@ Only Terraform 0.12 is supported. ## Usage -- **Private Bucket** +### Private bucket with versioning enabled ```hcl module "s3_bucket" { source = "terraform-aws-modules/s3-bucket/aws" - bucket = "s3-tf-example-versioning" + bucket = "my-s3-bucket" acl = "private" - versioning_inputs = [ - { - enabled = true - mfa_delete = null - }, - ] + versioning = { + enabled = true + } + } ``` -## Examples: +## Conditional creation -* [S3-CORS](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/s3-cors) -* [S3-Lifecycle-Rules](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/s3-lifecycle-rules) -* [S3-Logging](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/s3-logging) -* [S3-Replication](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/s3-replication) -* [S3-Versioning](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/s3-versioning) -* [S3-Website](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/s3-website) +Sometimes you need to have a way to create S3 resources conditionally but Terraform does not allow to use `count` inside `module` block, so the solution is to specify argument `create_bucket`. -## Inputs notes -``` -The Terraform "aws_s3_bucket" resource has some nested configuration blocks and this was translated -to this module as lists of objects. Each configuration block was renamed as it follows: -_inputs -``` - - -``` -module "aws_s3_bucket" { - source = "../.." - bucket = "s3-tf-example-logging" - acl = "private" +```hcl +# This S3 bucket will not be created +module "s3_bucket" { + source = "terraform-aws-modules/s3-bucket/aws" - logging_inputs = [ - { - target_bucket = "s3-tf-example-logger" - target_prefix = "log/" - }, - ] -``` -The **logging_inputs** list will be converted to a **logging** configuration block: -``` -logging { - target_bucket = "s3-tf-example-logger" - target_prefix = "log/" + create_bucket = false + # ... omitted } ``` +## Examples: + +* [Complete](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/complete) - Complete S3 bucket with most of supported features enabled +* [Cross-Region Replication](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/s3-replication) - S3 bucket with Cross-Region Replication (CRR) enabled + ## Inputs @@ -96,32 +67,33 @@ logging { | acl | (Optional) The canned ACL to apply. Defaults to 'private'. | string | `"private"` | no | | bucket | (Optional, Forces new resource) The name of the bucket. If omitted, Terraform will assign a random, unique name. | string | `"null"` | no | | bucket\_prefix | (Optional, Forces new resource) Creates a unique bucket name beginning with the specified prefix. Conflicts with bucket. | string | `"null"` | no | -| cors\_rule\_inputs | | object | `"null"` | no | -| force\_destroy | (Optional, Default:false ) A boolean that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable. | string | `"false"` | no | -| lifecycle\_rule\_inputs | | object | `"null"` | no | -| logging\_inputs | | object | `"null"` | no | -| object\_lock\_configuration\_inputs | | object | `"null"` | no | +| cors\_rule | Map containing a rule of Cross-Origin Resource Sharing. | any | `{}` | no | +| create\_bucket | Controls if S3 bucket should be created | bool | `"true"` | no | +| force\_destroy | (Optional, Default:false ) A boolean that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable. | bool | `"false"` | no | +| lifecycle\_rule | List of maps containing configuration of object lifecycle management. | any | `[]` | no | +| logging | Map containing access bucket logging configuration. | map(string) | `{}` | no | +| object\_lock\_configuration | Map containing S3 object locking configuration. | any | `{}` | no | | policy | (Optional) A valid bucket policy JSON document. Note that if the policy document is not specific enough (but still valid), Terraform may view the policy as constantly changing in a terraform plan. In this case, please make sure you use the verbose/specific version of the policy. For more information about building AWS IAM policy documents with Terraform, see the AWS IAM Policy Document Guide. | string | `"null"` | no | | region | (Optional) If specified, the AWS region this bucket should reside in. Otherwise, the region used by the callee. | string | `"null"` | no | -| replication\_configuration\_inputs | | object | `"null"` | no | +| replication\_configuration | Map containing cross-region replication configuration. | any | `{}` | no | | request\_payer | (Optional) Specifies who should bear the cost of Amazon S3 data transfer. Can be either BucketOwner or Requester. By default, the owner of the S3 bucket would incur the costs of any data transfer. See Requester Pays Buckets developer guide for more information. | string | `"null"` | no | -| server\_side\_encryption\_configuration\_inputs | | object | `"null"` | no | -| tags | (Optional) A mapping of tags to assign to the bucket. | map | `{}` | no | -| versioning\_inputs | | object | `"null"` | no | -| website\_inputs | | object | `"null"` | no | +| server\_side\_encryption\_configuration | Map containing server-side encryption configuration. | any | `{}` | no | +| tags | (Optional) A mapping of tags to assign to the bucket. | map(string) | `{}` | no | +| versioning | Map containing versioning configuration. | map(string) | `{}` | no | +| website | Map containing static web-site hosting or redirect configuration. | map(string) | `{}` | no | ## Outputs | Name | Description | |------|-------------| -| arn | The ARN of the bucket. Will be of format arn:aws:s3:::bucketname. | -| bucket\_domain\_name | The bucket domain name. Will be of format bucketname.s3.amazonaws.com. | -| bucket\_regional\_domain\_name | The bucket region-specific domain name. The bucket domain name including the region name, please refer here for format. Note: The AWS CloudFront allows specifying S3 region-specific endpoint when creating S3 origin, it will prevent redirect issues from CloudFront to S3 Origin URL. | -| hosted\_zone\_id | The Route 53 Hosted Zone ID for this bucket's region. | -| id | The name of the bucket. | -| region | The AWS region this bucket resides in. | -| website\_domain | The domain of the website endpoint, if the bucket is configured with a website. If not, this will be an empty string. This is used to create Route 53 alias records. | -| website\_endpoint | The website endpoint, if the bucket is configured with a website. If not, this will be an empty string. | +| this\_s3\_bucket\_arn | The ARN of the bucket. Will be of format arn:aws:s3:::bucketname. | +| this\_s3\_bucket\_bucket\_domain\_name | The bucket domain name. Will be of format bucketname.s3.amazonaws.com. | +| this\_s3\_bucket\_bucket\_regional\_domain\_name | The bucket region-specific domain name. The bucket domain name including the region name, please refer here for format. Note: The AWS CloudFront allows specifying S3 region-specific endpoint when creating S3 origin, it will prevent redirect issues from CloudFront to S3 Origin URL. | +| this\_s3\_bucket\_hosted\_zone\_id | The Route 53 Hosted Zone ID for this bucket's region. | +| this\_s3\_bucket\_id | The name of the bucket. | +| this\_s3\_bucket\_region | The AWS region this bucket resides in. | +| this\_s3\_bucket\_website\_domain | The domain of the website endpoint, if the bucket is configured with a website. If not, this will be an empty string. This is used to create Route 53 alias records. | +| this\_s3\_bucket\_website\_endpoint | The website endpoint, if the bucket is configured with a website. If not, this will be an empty string. | diff --git a/examples/complete/README.md b/examples/complete/README.md new file mode 100644 index 00000000..55ce9d44 --- /dev/null +++ b/examples/complete/README.md @@ -0,0 +1,40 @@ +# Complete S3 bucket with most of supported features enabled + +Configuration in this directory creates S3 bucket which demos such capabilities: +- static web-site hosting +- access logging +- versioning +- CORS +- lifecycle rules +- server-side encryption +- object locking + +Please check [S3 replication example](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/complete) to see Cross-Region Replication (CRR) supported by this module. + +## Usage + +To run this example you need to execute: + +```bash +$ terraform init +$ terraform plan +$ terraform apply +``` + +Note that this example may create resources which cost money. Run `terraform destroy` when you don't need these resources. + + +## Outputs + +| Name | Description | +|------|-------------| +| this\_s3\_bucket\_arn | The ARN of the bucket. Will be of format arn:aws:s3:::bucketname. | +| this\_s3\_bucket\_bucket\_domain\_name | The bucket domain name. Will be of format bucketname.s3.amazonaws.com. | +| this\_s3\_bucket\_bucket\_regional\_domain\_name | The bucket region-specific domain name. The bucket domain name including the region name, please refer here for format. Note: The AWS CloudFront allows specifying S3 region-specific endpoint when creating S3 origin, it will prevent redirect issues from CloudFront to S3 Origin URL. | +| this\_s3\_bucket\_hosted\_zone\_id | The Route 53 Hosted Zone ID for this bucket's region. | +| this\_s3\_bucket\_id | The name of the bucket. | +| this\_s3\_bucket\_region | The AWS region this bucket resides in. | +| this\_s3\_bucket\_website\_domain | The domain of the website endpoint, if the bucket is configured with a website. If not, this will be an empty string. This is used to create Route 53 alias records. | +| this\_s3\_bucket\_website\_endpoint | The website endpoint, if the bucket is configured with a website. If not, this will be an empty string. | + + diff --git a/examples/complete/main.tf b/examples/complete/main.tf new file mode 100644 index 00000000..da70ff32 --- /dev/null +++ b/examples/complete/main.tf @@ -0,0 +1,133 @@ +resource "random_pet" "this" { + length = 2 +} + +resource "aws_kms_key" "objects" { + description = "KMS key is used to encrypt bucket objects" + deletion_window_in_days = 7 +} + +module "log_bucket" { + source = "../../" + bucket = "logs-${random_pet.this.id}" + acl = "log-delivery-write" + force_destroy = true +} + +module "s3_bucket" { + source = "../../" + + bucket = "s3-bucket-${random_pet.this.id}" + acl = "private" + force_destroy = true + + tags = { + Owner = "Anton" + } + + versioning = { + enabled = true + } + + website = { + index_document = "index.html" + error_document = "error.html" + routing_rules = jsonencode([{ + Condition : { + KeyPrefixEquals : "docs/" + }, + Redirect : { + ReplaceKeyPrefixWith : "documents/" + } + }]) + + } + + logging = { + target_bucket = module.log_bucket.this_s3_bucket_id + target_prefix = "log/" + } + + cors_rule = { + allowed_methods = ["PUT", "POST"] + allowed_origins = ["https://modules.tf", "https://terraform-aws-modules.modules.tf"] + allowed_headers = ["*"] + expose_headers = ["ETag"] + max_age_seconds = 3000 + } + + lifecycle_rule = [ + { + id = "log" + enabled = true + prefix = "log/" + + tags = { + rule = "log" + autoclean = "true" + } + + transition = [ + { + days = 30 + storage_class = "ONEZONE_IA" + }, { + days = 60 + storage_class = "GLACIER" + } + ] + + expiration = { + days = 90 + } + + noncurrent_version_expiration = { + days = 30 + } + }, + { + id = "log1" + enabled = true + prefix = "log1/" + abort_incomplete_multipart_upload_days = 7 + + noncurrent_version_transition = [ + { + days = 30 + storage_class = "STANDARD_IA" + }, + { + days = 60 + storage_class = "ONEZONE_IA" + }, + { + days = 90 + storage_class = "GLACIER" + }, + ] + + noncurrent_version_expiration = { + days = 300 + } + }, + ] + + server_side_encryption_configuration = { + rule = { + apply_server_side_encryption_by_default = { + kms_master_key_id = aws_kms_key.objects.arn + sse_algorithm = "aws:kms" + } + } + } + + object_lock_configuration = { + object_lock_enabled = "Enabled" + rule = { + default_retention = { + mode = "COMPLIANCE" + years = 5 + } + } + } +} diff --git a/examples/complete/outputs.tf b/examples/complete/outputs.tf new file mode 100644 index 00000000..d7a44021 --- /dev/null +++ b/examples/complete/outputs.tf @@ -0,0 +1,39 @@ +output "this_s3_bucket_id" { + description = "The name of the bucket." + value = module.s3_bucket.this_s3_bucket_id +} + +output "this_s3_bucket_arn" { + description = "The ARN of the bucket. Will be of format arn:aws:s3:::bucketname." + value = module.s3_bucket.this_s3_bucket_arn +} + +output "this_s3_bucket_bucket_domain_name" { + description = "The bucket domain name. Will be of format bucketname.s3.amazonaws.com." + value = module.s3_bucket.this_s3_bucket_bucket_domain_name +} + +output "this_s3_bucket_bucket_regional_domain_name" { + description = "The bucket region-specific domain name. The bucket domain name including the region name, please refer here for format. Note: The AWS CloudFront allows specifying S3 region-specific endpoint when creating S3 origin, it will prevent redirect issues from CloudFront to S3 Origin URL." + value = module.s3_bucket.this_s3_bucket_bucket_regional_domain_name +} + +output "this_s3_bucket_hosted_zone_id" { + description = "The Route 53 Hosted Zone ID for this bucket's region." + value = module.s3_bucket.this_s3_bucket_hosted_zone_id +} + +output "this_s3_bucket_region" { + description = "The AWS region this bucket resides in." + value = module.s3_bucket.this_s3_bucket_region +} + +output "this_s3_bucket_website_endpoint" { + description = "The website endpoint, if the bucket is configured with a website. If not, this will be an empty string." + value = module.s3_bucket.this_s3_bucket_website_endpoint +} + +output "this_s3_bucket_website_domain" { + description = "The domain of the website endpoint, if the bucket is configured with a website. If not, this will be an empty string. This is used to create Route 53 alias records. " + value = module.s3_bucket.this_s3_bucket_website_domain +} diff --git a/examples/s3-cors/main.tf b/examples/s3-cors/main.tf deleted file mode 100644 index c2db56eb..00000000 --- a/examples/s3-cors/main.tf +++ /dev/null @@ -1,34 +0,0 @@ -variable "region" { - default = "us-west-2" -} - -# Configure the AWS Provider -provider "aws" { - region = var.region -} - -// Calling module: - -module "aws_s3_bucket" { - source = "../.." - bucket = "s3-tf-example-cors" - acl = "private" - - cors_rule_inputs = [ - { - allowed_headers = ["*"] - allowed_methods = ["PUT", "POST"] - allowed_origins = ["https://s3-website-test.hashicorp.com", "https://s3-website-test.hashicorp.io"] - expose_headers = ["ETag"] - max_age_seconds = 3000 - }, - { - allowed_headers = ["*"] - allowed_methods = ["GET"] - allowed_origins = ["https://s3-website-test.hashicorp.io"] - expose_headers = ["ETag"] - max_age_seconds = 3000 - }, - ] - -} \ No newline at end of file diff --git a/examples/s3-lifecycle-rule/main.tf b/examples/s3-lifecycle-rule/main.tf deleted file mode 100644 index 0ba08e17..00000000 --- a/examples/s3-lifecycle-rule/main.tf +++ /dev/null @@ -1,67 +0,0 @@ -variable "region" { - default = "us-west-2" -} - -# Configure the AWS Provider -provider "aws" { - region = var.region -} - -// Calling module: - -module "aws_s3_bucket" { - source = "../.." - bucket = "s3-tf-example-lifecycle" - acl = "private" - - lifecycle_rule_inputs = [{ - id = "log" - enabled = true - prefix = "log/" - abort_incomplete_multipart_upload_days = null - tags = { - "rule" = "log" - "autoclean" = "true" - } - - expiration_inputs = [{ - days = 90 - date = null - expired_object_delete_marker = null - }, - ] - transition_inputs = [] - noncurrent_version_transition_inputs = [] - noncurrent_version_expiration_inputs = [] - - }, - { - id = "log1" - enabled = true - prefix = "log1/" - abort_incomplete_multipart_upload_days = null - tags = { - "rule" = "log1" - "autoclean" = "true" - } - - expiration_inputs = [] - transition_inputs = [] - noncurrent_version_transition_inputs = [ - { - days = 30 - storage_class = "STANDARD_IA" - }, - { - days = 60 - storage_class = "ONEZONE_IA" - }, - { - days = 90 - storage_class = "GLACIER" - }, - ] - noncurrent_version_expiration_inputs = [] - }, - ] -} \ No newline at end of file diff --git a/examples/s3-logging/main.tf b/examples/s3-logging/main.tf deleted file mode 100644 index 552b38ce..00000000 --- a/examples/s3-logging/main.tf +++ /dev/null @@ -1,32 +0,0 @@ -variable "region" { - default = "us-west-2" -} - -# Configure the AWS Provider -provider "aws" { - region = var.region -} - -// Calling module: - -module "log_bucket" { - source = "../.." - bucket = "s3-tf-example-logger" - acl = "log-delivery-write" - - -} - -module "aws_s3_bucket" { - source = "../.." - bucket = "s3-tf-example-logging" - acl = "private" - - logging_inputs = [ - { - target_bucket = "s3-tf-example-logger" - target_prefix = "log/" - }, - ] - -} diff --git a/examples/s3-replication/README.md b/examples/s3-replication/README.md new file mode 100644 index 00000000..c4a10321 --- /dev/null +++ b/examples/s3-replication/README.md @@ -0,0 +1,33 @@ +# S3 bucket with Cross-Region Replication (CRR) enabled + +Configuration in this directory creates S3 bucket in one region and configures CRR to another bucket in another region. + +Please check [complete example](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/complete) to see all other features supported by this module. + +## Usage + +To run this example you need to execute: + +```bash +$ terraform init +$ terraform plan +$ terraform apply +``` + +Note that this example may create resources which cost money. Run `terraform destroy` when you don't need these resources. + + +## Outputs + +| Name | Description | +|------|-------------| +| this\_s3\_bucket\_arn | The ARN of the bucket. Will be of format arn:aws:s3:::bucketname. | +| this\_s3\_bucket\_bucket\_domain\_name | The bucket domain name. Will be of format bucketname.s3.amazonaws.com. | +| this\_s3\_bucket\_bucket\_regional\_domain\_name | The bucket region-specific domain name. The bucket domain name including the region name, please refer here for format. Note: The AWS CloudFront allows specifying S3 region-specific endpoint when creating S3 origin, it will prevent redirect issues from CloudFront to S3 Origin URL. | +| this\_s3\_bucket\_hosted\_zone\_id | The Route 53 Hosted Zone ID for this bucket's region. | +| this\_s3\_bucket\_id | The name of the bucket. | +| this\_s3\_bucket\_region | The AWS region this bucket resides in. | +| this\_s3\_bucket\_website\_domain | The domain of the website endpoint, if the bucket is configured with a website. If not, this will be an empty string. This is used to create Route 53 alias records. | +| this\_s3\_bucket\_website\_endpoint | The website endpoint, if the bucket is configured with a website. If not, this will be an empty string. | + + diff --git a/examples/s3-replication/iam.tf b/examples/s3-replication/iam.tf new file mode 100644 index 00000000..49475ee6 --- /dev/null +++ b/examples/s3-replication/iam.tf @@ -0,0 +1,65 @@ +resource "aws_iam_role" "replication" { + name = "s3-bucket-replication-${random_pet.this.id}" + + assume_role_policy = <