Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added new seadfarmer module for Question and Answering Gen AI construct #81

Merged
merged 31 commits into from
May 23, 2024

Conversation

saikatak
Copy link
Contributor

@saikatak saikatak commented May 8, 2024

Describe your changes

Added new seadfarmer module for Question and Answering Gen AI construct

Issue ticket number and link

Checklist before requesting a review

  • I updated CHANGELOG.MD with a description of my changes
  • If the change was to a module, I ran the code validation script (scripts/validate.sh)
  • If the change was to a module, I have added thorough tests
  • If the change was to a module, I have added/updated the module's README.md
  • If a module was added, I added a reference to the module to the repository's README.md
  • I verified that my code deploys successfully using seedfarmer apply

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: 5ed92ce
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: 1bf11ab
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: 2422dd7
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

)

# get an existing userpool
cognito_pool_id = cognito_pool_id
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Congnito mandatory? Can we add a parameter to support both IAM and Cognito?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As of today cognito is mandatory to be used with this construct. Created a feature request on the construct to make cognito user pool id as optional parameter

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

path: modules/fmops/qna-rag
parameters:
- name: cognito-pool-id
value: us-east-1-XXXXX
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we need an IDF module for Cognito. Can you move the manifest in examples/manifests/ since this requires user input/pre-setup Conginto user pool? We can move it back once we have congnito module/or IAM-only auth mode.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the manifest to examples

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: 35858b6
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: ec7510e
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: 3b8f21b
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

vpc_id = os.getenv(_param("VPC_ID"))
cognito_pool_id = os.getenv(_param("COGNITO_POOL_ID"))
os_domain_endpoint = os.getenv(_param("OS_DOMAIN_ENDPOINT"))
os_security_group_id = os.getenv(_param("OS_SECURITY_GROUP_ID"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor query - can you please add ability to provide input asset bucket & enable observability (ref params here)?

kukushking
kukushking previously approved these changes May 13, 2024
Copy link
Contributor

@kukushking kukushking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you!

I added a few minor comments, also as discussed this covers the consumer part, and in the next PRs would be great to see also the ingestion/embeddings generation for a complete E2E flow. Thanks!

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: 53737d7
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: 775d7c1
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: e21a652
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: 45459f9
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: e36916a
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: 87c7b65
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: 301206e
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: a24c5c7
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

kukushking
kukushking previously approved these changes May 22, 2024
Copy link
Contributor

@kukushking kukushking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thank you!

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: d3ab066
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

kukushking
kukushking previously approved these changes May 22, 2024
@@ -51,3 +51,25 @@ parameters:
value: 30
- name: removal-policy
value: DESTROY
---
name: opensearch
path: git::https://github.com/awslabs/idf-modules.git//modules/storage/opensearch/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should lock this version to a release of IDF:

git::https://github.com/awslabs/idf-modules.git//modules/storage/opensearch?ref=release/1.7.0&depth=1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the manifest

@@ -0,0 +1,22 @@
---
name: opensearch
path: git::https://github.com/awslabs/idf-modules.git//modules/storage/opensearch/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should lock this version to a release of IDF:

git::https://github.com/awslabs/idf-modules.git//modules/storage/opensearch?ref=release/1.7.0&depth=1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the manifest

)
os_security_group.add_ingress_rule(
peer=security_group,
connection=ec2.Port.tcp(443),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this port number coming from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the port on which open search is listening, Thus allowing traffic to Open search from lambda on port 443. Can make this configurable if that makes sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made the port configurable parameter with default value as 443

@malachi-constant
Copy link
Contributor

AWS CodeBuild CI Report

  • CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
  • Commit ID: ad0e94d
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@kukushking kukushking merged commit cb9844f into awslabs:main May 23, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants