This repository contains a Terraform module for creating an EKS cluster and all the necessary infrastructure to install Jenkins X via jx boot.
The module generates for this purpose a templated jx-requirements.yml file for jx boot.
The module makes use of the Terraform EKS cluster Module.
- What is a Terraform module
- How do you use this module
- FAQ: Frequently Asked Questions
- Development
- How can I contribute
A Terraform module refers to a self-contained package of Terraform configurations that are managed as a group. For more information about modules refer to the Terraform documentation.
This Terraform module allows you to create an EKS cluster for installation of Jenkins X. You need the following binaries locally installed and configured on your PATH:
terraform(~> 0.12.0)kubectl(>=1.10)aws-iam-authenticator
A default Jenkins X ready cluster can be provisioned by creating a main.tf file in an empty directory with the following content:
module "eks-jx" {
source = "jenkins-x/eks-jx/aws"
}
output "vault_user_id" {
value = module.eks-jx.vault_user_id
description = "The Vault IAM user id"
}
output "vault_user_secret" {
value = module.eks-jx.vault_user_secret
description = "The Vault IAM user secret"
}
Due to the Vault issue 7450, this Terraform module needs for now to create a new IAM user for installing Vault.
It also creates an IAM access key whose id and secret are defined in the output above.
You need the id and secret for running jx boot.
If you do not want Terraform to create a new IAM user or you do not have permissions to create one, you need to provide the name of an existing IAM user.
module "eks-jx" {
source = "jenkins-x/eks-jx/aws"
vault_user="<your_vault_iam_username>"
}The IAM user does not need any permissions attached to it. For more information, refer to Configuring Vault for EKS in the Jenkins X documentation.
Once you have your initial configuration, you can apply it by running:
terraform init
terraform applyThis creates an EKS cluster with all possible configuration options defaulted.
You then need to export the environment variables VAULT_AWS_ACCESS_KEY_ID and VAULT_AWS_SECRET_ACCESS_KEY.
export VAULT_AWS_ACCESS_KEY_ID=$(terraform output vault_user_id)
export VAULT_AWS_SECRET_ACCESS_KEY=$(terraform output vault_user_secret)If you specified vault_user you need to provide the access key id and secret for the specified user.
The following sections provide a full list of configuration in- and output variables.
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| apex_domain | The main domain to either use directly or to configure a subdomain from | string |
"" |
no |
| cluster_name | Variable to provide your desired name for the cluster. The script will create a random name if this is empty | string |
"" |
no |
| cluster_version | Variable to provide your desired Kubernetes version for the cluster. | string |
"1.15" |
no |
| create_and_configure_subdomain | Flag to create an NS record set for the subdomain in the apex domain's Hosted Zone | bool |
false |
no |
| desired_node_count | The number of worker nodes to use for the cluster | number |
3 |
no |
| enable_external_dns | Flag to enable or disable External DNS in the final jx-requirements.yml file |
bool |
false |
no |
| enable_logs_storage | Flag to enable or disable long term storage for logs | bool |
true |
no |
| enable_reports_storage | Flag to enable or disable long term storage for reports | bool |
true |
no |
| enable_repository_storage | Flag to enable or disable the repository bucket storage | bool |
true |
no |
| enable_tls | Flag to enable TLS in the final jx-requirements.yml file |
bool |
false |
no |
| max_node_count | The maximum number of worker nodes to use for the cluster | number |
5 |
no |
| min_node_count | The minimum number of worker nodes to use for the cluster | number |
3 |
no |
| node_machine_type | The instance type to use for the cluster's worker nodes | string |
"m5.large" |
no |
| production_letsencrypt | Flag to use the production environment of letsencrypt in the jx-requirements.yml file |
bool |
false |
no |
| region | The region to create the resources into | string |
"us-east-1" |
no |
| subdomain | The subdomain to be added to the apex domain. If subdomain is set, it will be appended to the apex domain in jx-requirements-eks.yml file |
string |
"" |
no |
| tls_email | The email to register the LetsEncrypt certificate with. Added to the jx-requirements.yml file |
string |
"" |
no |
| vault_user | The AWS IAM Username whose credentials will be used to authenticate the Vault pods against AWS | string |
"" |
no |
| vpc_cidr_block | The vpc CIDR block | string |
"10.0.0.0/16" |
no |
| vpc_name | The name of the VPC to be created for the cluster | string |
"tf-vpc-eks" |
no |
| vpc_subnets | The subnet CIDR block to use in the created VPC | list(string) |
[ |
no |
| Name | Description |
|---|---|
| cert_manager_iam_role | The IAM Role that the Cert Manager pod will assume to authenticate |
| cluster_name | The name of the created cluster |
| cm_cainjector_iam_role | The IAM Role that the CM CA Injector pod will assume to authenticate |
| controllerbuild_iam_role | The IAM Role that the ControllerBuild pod will assume to authenticate |
| external_dns_iam_role | The IAM Role that the External DNS pod will assume to authenticate |
| jxui_iam_role | The IAM Role that the Jenkins X UI pod will assume to authenticate |
| lts_logs_bucket | The bucket where logs from builds will be stored |
| lts_reports_bucket | The bucket where test reports will be stored |
| lts_repository_bucket | The bucket that will serve as artifacts repository |
| tekton_bot_iam_role | The IAM Role that the build pods will assume to authenticate |
| vault_dynamodb_table | The Vault DynamoDB table |
| vault_kms_unseal | The Vault KMS Key for encryption |
| vault_unseal_bucket | The Vault storage bucket |
| vault_user_id | The Vault IAM user id |
| vault_user_secret | The Vault IAM user secret |
You can choose to create S3 buckets for long term storage and enable them in the generated jx-requirements.yml file with enable_logs_storage, enable_reports_storage and enable_repository_storage.
During terraform apply the enabledS3 buckets are created, and the generated jx-requirements.yml will contain the following section:
storage:
logs:
enabled: ${enable_logs_storage}
url: s3://${logs_storage_bucket}
reports:
enabled: ${enable_reports_storage}
url: s3://${reports_storage_bucket}
repository:
enabled: ${enable_repository_storage}
url: s3://${repository_storage_bucket}Vault is used by Jenkins X for managing secrets. Part of this module's responsibilities is the creation of all resources required to run the Vault Operator. These resources are An S3 Bucket, a DynamoDB Table and a KMS Key.
You can enable ExternalDNS with the enable_external_dns variable. This modifies the generated jx-requirements.yml file to enable External DNS when running jx boot.
If enable_external_dns is true, additional configuration is required.
If you want to use a domain with an already existing Route 53 Hosted Zone, you can provide it through the apex_domain variable:
This domain will be configured in the resulting jx-requirements.yml file in the following section:
ingress:
domain: ${domain}
ignoreLoadBalancer: true
externalDNS: ${enable_external_dns}If you want to use a subdomain and have this module create and configure a new Hosted Zone with DNS delegation, you can provide the following variables:
subdomain: This subdomain is added to the apex domain and configured in the resulting jx-requirements.yml file.
create_and_configure_subdomain: This flag instructs the script to create a new Route53 Hosted Zone for your subdomain and configure DNS delegation with the apex domain.
By providing these variables, the script creates a new Route 53 HostedZone that looks like <subdomain>.<apex_domain>, then it delegates the resolving of DNS to the apex domain.
This is done by creating a NS RecordSet in the apex domain's Hosted Zone with the subdomain's HostedZone nameservers.
This ensures that the newly created HostedZone for the subdomain is instantly resolvable instead of having to wait for DNS propagation.
You can enable cert-manager to use TLS for your cluster through LetsEncrypt with the enable_tls variable.
LetsEncrypt has two environments, staging and production.
If you use staging, you will receive self-signed certificates, but you are not rate-limited, if you use the production environment, you receive certificates signed by LetsEncrypt, but you can be rate limited.
You can choose to use the production environment with the production_letsencrypt variable:
You need to provide a valid email to register your domain in LetsEncrypt with tls_email:
An output of applying this Terraform module is a jx-requirements.yml file in the current directory. This file can be used as input to Jenkins X Boot which is responsible for installing all the required Jenkins X components into the cluster created by this module.
Change into an empty directory and execute:
jx boot --requirements <path-to-jx-requirements.yml>❕ Remember you need to export VAULT_AWS_ACCESS_KEY_ID and VAULT_AWS_SECRET_ACCESS_KEY. See Cluster provisioning.
You are prompted for any further required configuration. The number of prompts depends on how much you have pre-configured via your Terraform variables.
The configuration, as seen in Cluster provisioning, is not suited for creating and maintaining a production Jenkins X cluster. The following is a list of considerations for a production use case.
-
Specify the version attribute of the module, for example:
module "jx" { source = "jenkins-x/eks-jx/aws" version = "1.0.0" # insert your configuration }
Specifying the version ensures that you are using a fixed version and that version upgrades cannot occur unintended.
-
Keep the Terraform configuration under version control by creating a dedicated repository for your cluster configuration or by adding it to an already existing infrastructure repository.
-
Setup a Terraform backend to securely store and share the state of your cluster. For more information refer to Configuring a Terraform backend.
A "backend" in Terraform determines how state is loaded and how an operation such as apply is executed. By default, Terraform uses the local backend, which keeps the state of the created resources on the local file system. This is problematic since sensitive information will be stored on disk and it is not possible to share state across a team. When working with AWS a good choice for your Terraform backend is the s3 backend which stores the Terraform state in an AWS S3 bucket. The examples directory of this repository contains configuration examples for using the s3 backed.
To use the s3 backend, you will need to create the bucket upfront. You need the S3 bucket as well as a Dynamo table for state locks. You can use terraform-aws-tfstate-backend to create these required resources.
You can find examples for different configurations in the examples folder.
Each example generates a valid jx-requirements.yml file that can be used to boot a Jenkins X cluster.
This module sets up a series of IAM Policies and Roles. These roles will be annotated into a few Kubernetes Service accounts.
This allows us to make use of IAM Roles for Sercive Accounts to set fine-grained permissions on a pod per pod basis.
There is no way to provide your own roles or define other Service Accounts by variables, but you can always modify the eks/terraform/jx/irsa.tf Terraform file.
At the moment, there is no release pipeline defined in jenkins-x.yml.
A Terraform release does not require building an artifact; only a tag needs to be created and pushed.
To make this task easier and there is a helper script release.sh which simplifies this process and creates the changelog as well:
./scripts/release.shThis can be executed on demand whenever a release is required. For the script to work, the environment variable $GH_TOKEN must be exported and reference a valid GitHub API token.
Contributions are very welcome! Check out the Contribution Guidelines for instructions.