Skip to content

Commit

Permalink
Enable CI/CD on DEV/PROD branches (#31)
Browse files Browse the repository at this point in the history
* MAJOR refactor and initial CI/CD pipeline

* trigger workflow

* Test workflow dispatch branch

* Update CI/CD pipeline fixes

* Update CI/CD pipeline fixes

* Update CI/CD pipeline fixes for CDK and libs

* Update CI/CD pipeline fixes for CDK npm fixes

* Update CI/CD pipeline fixes for CDK synth env-vars

* Update CI/CD pipeline fixes for CDK paths and envs

* Update CI/CD pipeline fixes for nextjs output folder

* Update CI/CD pipeline fixes for npm package-lock.json

* Update CI/CD pipeline fixes for CDK archives output

* Update README.md with assets, CI/CD and more details

* Minor index update to validate DEV/PROD changes
  • Loading branch information
san99tiago authored Feb 8, 2024
1 parent d486f0a commit 0b069e5
Show file tree
Hide file tree
Showing 39 changed files with 1,970 additions and 1,448 deletions.
212 changes: 212 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
name: deploy

on:
push:
branches: [ 'main', 'develop']

env:
AWS_DEFAULT_REGION: us-east-1
AWS_DEFAULT_OUTPUT: json

jobs:
code-quality:
name: Check coding standards
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- run: echo "Job triggered by ${{ github.event_name }} event."
- run: echo "Job running on a ${{ runner.os }} server hosted by GitHub."
- run: echo "Branch name is ${{ github.ref }} and repository is ${{ github.repository }}."
- name: Set up NodeJs
uses: actions/setup-node@v3
with:
node-version: "20"
- name: Run Prettier and Lint
run: |
cd src
npm ci
npm run prettier-check
npm run lint
build-nextjs:
name: Build NextJS Application
runs-on: ubuntu-latest
needs: code-quality
steps:
- uses: actions/checkout@v3
- name: Set up NodeJs
uses: actions/setup-node@v3
with:
node-version: "20"
- name: Build NextJs App to Output folder
run: bash build.sh
- name: Archive NextJS Output dir
uses: actions/upload-artifact@v3
with:
name: nextjs-output-dir
path: src/out

cdk-synth-diff:
name: CDK Synth & Diff
runs-on: ubuntu-latest
needs: build-nextjs
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout
steps:
- uses: actions/checkout@v3

- name: Dowload NextJS output folder
uses: actions/download-artifact@v3
with:
name: nextjs-output-dir
path: ./src/out

- name: Set up NodeJs
uses: actions/setup-node@v3
with:
node-version: "20"

- name: Install CDK and Modules
run: |
npm install -g aws-cdk
cd ./cdk
npm ci
# Same task with different secrets depending on the branch ref (dev vs prod deployments)
# Note: there might be better alternatives, but this is a workaround to deploy to both envs
- name: Configure AWS Credentials (DEV)
if: github.ref != 'refs/heads/main'
uses: aws-actions/configure-aws-credentials@v4
with:
aws-region: ${{ env.AWS_DEFAULT_REGION }}
role-to-assume: arn:aws:iam::${{ secrets.DEV_AWS_ACCOUNT_ID }}:role/${{ secrets.DEV_AWS_DEPLOY_ROLE }}
role-session-name: myGitHubActions
- name: Configure AWS Credentials (PROD)
if: github.ref == 'refs/heads/main'
uses: aws-actions/configure-aws-credentials@v4
with:
aws-region: ${{ env.AWS_DEFAULT_REGION }}
role-to-assume: arn:aws:iam::${{ secrets.PROD_AWS_ACCOUNT_ID }}:role/${{ secrets.PROD_AWS_DEPLOY_ROLE }}
role-session-name: myGitHubActions

- run: aws sts get-caller-identity

# Not 100% optimal, but does the work for setting deployment environments from branch
- name: Setup Deployment Environment from Branch
run: |
# To update the deployment environment based on the git branch
if [[ $GITHUB_REF == 'refs/heads/main' ]]; then
echo "DEPLOYMENT_ENVIRONMENT=prod" >> "$GITHUB_ENV"
echo "Deployment environment is PROD"
else
echo "DEPLOYMENT_ENVIRONMENT=dev" >> "$GITHUB_ENV"
echo "Deployment environment is DEV"
fi
- name: CDK Synth
run: |
cd ./cdk
cdk synth
- name: CDK Diff
run: |
cd ./cdk
cdk diff
- name: Archive CDK Synth results (no assets)
uses: actions/upload-artifact@v3
with:
name: cdk-synth-folder
path: |
./cdk/cdk.out
!./cdk/cdk.out/asset.*
retention-days: 1

iac-checkov:
name: IaC Checkov Validations
runs-on: ubuntu-latest
needs: cdk-synth-diff
steps:
- uses: actions/checkout@v3

- name: Dowload CDK Synth results
uses: actions/download-artifact@v3
with:
name: cdk-synth-folder
path: ./cdk-synth-output-folder

- name: Display files in the output folder
run: tree
working-directory: ./cdk-synth-output-folder

- name: Run Checkov action
id: checkov
uses: bridgecrewio/checkov-action@v12
with:
directory: cdk-synth-output-folder/
framework: cloudformation
soft_fail: true # optional: do not return an error code if there are failed checks
skip_check: CKV_AWS_2 # optional: skip a specific check_id. can be comma separated list
quiet: true # optional: display only failed checks

cdk-deploy:
name: Deploy CDK
runs-on: ubuntu-latest
needs:
- iac-checkov
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout
steps:
- uses: actions/checkout@v3

- name: Dowload NextJS output folder
uses: actions/download-artifact@v3
with:
name: nextjs-output-dir
path: ./src/out

- name: Set up NodeJs
uses: actions/setup-node@v3
with:
node-version: "20"

- name: Install CDK and Modules
run: |
npm install -g aws-cdk
cd ./cdk
npm ci
# Same task with different secrets depending on the branch ref (dev vs prod deployments)
# Note: there might be better alternatives, but this is a workaround to deploy to both envs
- name: Configure AWS Credentials (DEV)
if: github.ref != 'refs/heads/main'
uses: aws-actions/configure-aws-credentials@v4
with:
aws-region: ${{ env.AWS_DEFAULT_REGION }}
role-to-assume: arn:aws:iam::${{ secrets.DEV_AWS_ACCOUNT_ID }}:role/${{ secrets.DEV_AWS_DEPLOY_ROLE }}
role-session-name: myGitHubActions
- name: Configure AWS Credentials (PROD)
if: github.ref == 'refs/heads/main'
uses: aws-actions/configure-aws-credentials@v4
with:
aws-region: ${{ env.AWS_DEFAULT_REGION }}
role-to-assume: arn:aws:iam::${{ secrets.PROD_AWS_ACCOUNT_ID }}:role/${{ secrets.PROD_AWS_DEPLOY_ROLE }}
role-session-name: myGitHubActions

# Not 100% optimal, but does the work for setting deployment environments from branch
- name: Setup Deployment Environment from Branch
run: |
# To update the deployment environment based on the git branch
if [[ $GITHUB_REF == 'refs/heads/main' ]]; then
echo "DEPLOYMENT_ENVIRONMENT=prod" >> "$GITHUB_ENV"
echo "Deployment environment is PROD"
else
echo "DEPLOYMENT_ENVIRONMENT=dev" >> "$GITHUB_ENV"
echo "Deployment environment is DEV"
fi
# NOTE: for now no manual approvals are required
- name: Deploy to AWS
run: bash deploy.sh $DEPLOYMENT_ENVIRONMENT
84 changes: 57 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,56 +1,86 @@
# :scroll: SAN99TIAGO-CV :scroll:
# SAN99TIAGO-CV

Source code and Infrastructure for my personal [san99tiago.com](https://san99tiago.com) CV-like website.
Welcome to the _behind-the-scenes_ of [san99tiago.com](https://san99tiago.com) , my personal CV website!

The solution was developed as a Single Page Application website on top of [NextJS](https://nextjs.org) (production-grade framework on top of React), and deployed on AWS with an S3 bucket and a CDN on top of CloudFront.
> I've proudly opened up the source-code, CI/CD, and infrastructure as code of my project to the world. Why? Because sharing is caring! Whether you're a seasoned developer looking for inspiration or a beginner eager to learn, this project could be helpful for you.
## Diagrams
## AWS Architecture

The solution's diagrams can be divided into 2 categories:
<img src="assets/san99tiago_cv_aws_architecture.png" width=90%> <br>

- AWS Architecture Diagram (main AWS resources and solution)
- DNS Workflow Diagram [Dev/Prod] (how DNS/R53 is configured)
This awesome project was developed as a Single Page Application website on top of:

### AWS Architecture Diagram
- Source Code on [NextJS](https://nextjs.org): Production-grade Frontend framework on top of React.
- Infrastructure and Resources on [Amazon Web Services](https://aws.amazon.com): biggest cloud-computing provider.
- Amazon S3 for hosting/deploying the static website files (Object Storage).
- CloudFront to distribute the site as a Content Delivery Network (CDN).
- Route 53 for the DNS management for top level domain and subdomains.
- AWS Certificate Manager for the SSL/TLS certificate and security lifecycle.
- CloudWatch for observability on top of the requests and application management.
- Infrastructure as Code on [AWS Cloud Development Kit](https://aws.amazon.com/cdk/): IaC abstraction of top of common programming languages (in this case built with TypeScript).
- Software Development Life Cycle: Leverages [GitHub Actions](https://docs.github.com/en/actions) with CI/CD automation that is able to update the AWS solution E2E on both DEV/PROD accounts.
- Unit Tests: vary depending on the project's scope.
- Integration/Load Tests: built on top of [Locust Framework](https://locust.io): Modern load testing tool.

The AWS infrastructure solution is deployed with CDK-TypeScript with the resources defined on the `cdk` folder:
## Deployment

<img src="assets/san99tiago_cv_aws_architecture.png" width=90%> <br>
The CI/CD automation is built on top of GitHub Actions and GitHub Workflows. It is driven by the branch as follows:

- `main`: reserved for PROD deployments.
- `develop`: reserved for DEV deployments.
- Other Branches: can be created but will NOT trigger the CI/CD.

<img src="assets/san99tiago-cv-cicd.png" width=90%> <br>

To dive deeper into the deployment scripts, please explore the following files:

### DNS Workflow Diagram [Dev/Prod]
- [`.github/workflows/deploy.yml`](.github/workflows/deploy.yml): Definition of the GitHub Workflow for the pipeline that will trigger the deployments.
- [`build.sh`](build.sh): Bash file to generate the NextJS static artifact (export) for the Single Page Application:
- Outputs: `./out/` folder with the static files inside.
- [`deploy.sh`](deploy.sh)
- Run with: `bash deploy.sh <environment>` (set environment to `prod` or `dev` or leverage CI/CD).
- Behavior: deploys the solution based on current AWS Profile/Credentials and the given environment.

## DNS Workflow Diagram [Dev/Prod]

The DNS workflow is designed for a multi-account deployment with DEV/PROD environments. The idea is to have the following final endpoints and test any change on `DEV` environment prior to the `PROD` deployment:

- DEV: [dev.san99tiago.com](https://dev.san99tiago.com)
- PROD: [san99tiago.com](https://san99tiago.com)
- PROD: [san99tiago.com](https://san99tiago.com) --> Always active
- DEV: [dev.san99tiago.com](https://dev.san99tiago.com) --> Only active for tests/validations

To achieve these multiple DNS environments, 2 independent AWS accounts are used for the deployments (DEV/PROD), and the `dev.san99tiago.com` is delegated as a Route 53 Sub-Domain Hosted Zone in the `DEV` account.
To achieve these multiple DNS environments, 2 independent AWS accounts are used for the deployments (DEV/PROD), and the [dev.san99tiago.com](https://dev.san99tiago.com) is delegated as a Route 53 Sub-Domain Hosted Zone in the `DEV` account.

<img src="assets/dns_multi_account_san99tiago_com.png" width=90%> <br>

## Deployment

Currently, there is no automatic CI/CD deployment based on the repository branch (future state).
## Important Remarks

However, the deployments are done with the following files:
### Route 53 Hosted Zones and CloudFront Distribution

- [`build.sh`](build.sh): Bash file to generate the NextJS static artifact (export) for the Single Page Application:
- Run with: `bash build.sh` .
- Outputs: `./out/` folder with the static files inside.
- [`deploy.sh`](deploy.sh)
- Run with: `bash deploy.sh <environment>` (set environment to `prod` or `dev`).
- Behavior: deploys the solution based on current AWS Profile/Credentials and the given environment.
As I have already configured the Route 53 Hosted Zones on the target deployment accounts (dev/prod domains), the ACM Certificate validation process (the one attached to the CloudFront distribution) is automatic during the CDK/CloudFormation deployment. If the Hosted Zones were managed in other AWS accounts, the validation of the certificates would need to be done during the deployment manually in the Hosted Zone corresponding to the domain as a DNS validation to double check that we own the domain.

## Destroy
### Destroy Process (Only Manually)

If destroys are needed, we have 2 options:

- Directly delete the CloudFormation Stack (either via console or CLI).
- Setting an environment variable of the target environment, for example: `export DEPLOYMENT_ENVIRONMENT=dev` and run `cdk destroy`.

## Important Remarks
> Note: They are not supported in CI/CD to avoid potential destroy errors.
As I have already configured the Route 53 Hosted Zones on the target deployment accounts (dev/prod domains), the ACM Certificate validation process (the one attached to the CloudFront distribution) is automatic during the CDK/CloudFormation deployment. If the Hosted Zones were managed in other AWS accounts, the validation of the certificates would need to be done during the deployment manually in the Hosted Zone corresponding to the domain as a DNS validation to double check that we own the domain.
## Author

### Santiago Garcia Arango

<table border="1">
<tr>
<td>
<p align="center">Curious DevOps Engineer passionate about advanced cloud-based solutions and deployments in AWS. I am convinced that today's greatest challenges must be solved by people that love what they do.</p>
</td>
<td>
<p align="center"><img src="assets/SantiagoGarciaArango_AWS.png" width=80%></p>
</td>
</tr>
</table>

## LICENSE

Expand Down
Binary file added assets/SantiagoGarciaArango_AWS.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 0b069e5

Please sign in to comment.