From b61ee9f0e6366944e946443e7d73cf9ea508908e Mon Sep 17 00:00:00 2001 From: Rohit Sachdeva Date: Fri, 27 Sep 2024 12:33:46 -0500 Subject: [PATCH] Add test-e2e using localstack and update design This runs in CI! Added fr=ew more compoents to the design --- .github/workflows/rust.yml | 79 ++++++++++++++++++++++++++++++++++++++ DriveDeposits.drawio.svg | 4 ++ README.md | 9 ++++- justfile | 17 ++++++++ 4 files changed, 108 insertions(+), 1 deletion(-) create mode 100644 DriveDeposits.drawio.svg diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml index 03350bf..1460c45 100644 --- a/.github/workflows/rust.yml +++ b/.github/workflows/rust.yml @@ -103,6 +103,85 @@ jobs: - name: Run unit and integration tests run: just test + test-e2e: + needs: build + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + # Note: Rust is preinstalled on GitHub runners (example shows rustc 1.80.1, cargo 1.80.1 as of this commit) + # - name: Set up Rust + # uses: dtolnay/rust-toolchain@stable + - name: Check Rust version + run: | + echo "Installed rustc version:" + rustc --version + echo "Installed cargo version:" + cargo --version + + - name: Check Docker version + run: | + echo "Installed docker version:" + docker --version + + - name: Install Just + uses: extractions/setup-just@v2 + + - name: Check Just version + run: | + echo "Installed just version:" + just --version + + - name: Install protoc + uses: taiki-e/install-action@v2 + with: + tool: protoc@3.28.0 + + - name: Check protoc version + run: | + echo "Installed protoc version:" + protoc --version + + - name: Install Cargo Lambda + run: | + pip3 install cargo-lambda + echo "Installed cargo lambda version:" + cargo lambda --version + + - name: Run localstack detached + run: | + just localstack-start-detached + echo "Started localstack container" + + - name: Wait for LocalStack + run: | + while ! nc -z localhost 4566; do + echo "Waiting for LocalStack to be ready..." + sleep 1 + done + + - name: Install AWS SAM LOCAL CLI + run: | + sudo apt-get update + sudo apt-get install -y python3-pip + pip3 install aws-sam-cli-local + echo "Installed AWS SAM CLI version:" + samlocal --version + + - name: Run e2e tests; with deployment to localstack + env: + AWS_ACCESS_KEY_ID: test + AWS_SECRET_ACCESS_KEY: test + AWS_DEFAULT_REGION: us-west-2 + run: just test-e2e + + - name: Stop and remove localstack container + if: always() + run: | + just localstack-stop + echo "Stopped and removed localstack container" + # deploy: # runs-on: ubuntu-latest # needs: build diff --git a/DriveDeposits.drawio.svg b/DriveDeposits.drawio.svg new file mode 100644 index 0000000..f2ff315 --- /dev/null +++ b/DriveDeposits.drawio.svg @@ -0,0 +1,4 @@ + + + +
REST HTTP Gateway Server
gRPC Server
Calculation Engine
POST Request
Same POST request
GET Requests By Level
Hybrid testing tool: Check Command
          Same Data                  Transformations
{{api_gateway_host}}/by-level-for-portfolios/delta-growth?&order=desc&top_k=50


{{api_gateway_host}}/portfolios/{{aws_portfolio_uuid}}/by-level-for-banks/delta-growth?order=desc&top_k=4


{{api_gateway_host}}/portfolios/{{aws_portfolio_uuid}}/by-level-for-deposits/delta-growth?order=desc&top_k=4


{{api_gateway_host}}/portfolios/{{aws_portfolio_uuid}}/by-level-for-deposits/maturity-date?order=asc&top_k=17


Bank-Level
Portfolio Level
Portfolio Level
Log Group
Log Group
Async Tokio for sending Events to EventBridge
 \ No newline at end of file diff --git a/README.md b/README.md index e4d4a15..e2752e4 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,7 @@ - [Development Tool: LocalStack](#development-tool-localstack) - [Clean And Build](#clean-and-build) - [Configurations for DriveDeposits](#configurations-for-drivedeposits) +- [Current System Design](#current-system-design) - [Member crates in workspace](#member-crates-in-workspace) ### Domain Driven Terminology: @@ -291,7 +292,7 @@ a Lambda function connected to DynamoDB. `just post-calculate-portfolio-valid` -Follow up with [Data population and Querying](#data-population-and-querying) section to see how to query the data. +Follow up with [Data population](#data-population) section to see how to query the data. #### Command for AWS Lambda Invoke Check Directly @@ -508,6 +509,12 @@ AWS deployments. [Back to Table of Contents](#table-of-contents) +### Current System Design + +![DriveDeposits.drawio.svg](DriveDeposits.drawio.svg) + +[Back to Table of Contents](#table-of-contents) + ### Member crates in workspace See cargo workspace members: diff --git a/justfile b/justfile index 19ce5bc..ddd5cd8 100644 --- a/justfile +++ b/justfile @@ -269,8 +269,25 @@ localstack-start:localstack-build docker run -p 4566:4566 -p 4510-4559:4510-4559 \ -v ${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack \ -v /var/run/docker.sock:/var/run/docker.sock \ + --name localstack-container \ custom-localstack +localstack-start-detached:localstack-build + # Run a container from the built image, mapping the ports + docker run -d -p 4566:4566 -p 4510-4559:4510-4559 \ + -v ${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack \ + -v /var/run/docker.sock:/var/run/docker.sock \ + --name localstack-container \ + custom-localstack + +localstack-logs: + # View logs of the LocalStack container + docker logs localstack-container + +localstack-stop: + docker stop localstack-container && \ + docker rm localstack-container + # after localstack started # in localstack event source with bus localstack-build-drive-deposits-event-bus: