Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support custom code for Hugging Face SageMaker model deployment #313

Open
2 tasks
krokoko opened this issue Mar 11, 2024 · 0 comments
Open
2 tasks

support custom code for Hugging Face SageMaker model deployment #313

krokoko opened this issue Mar 11, 2024 · 0 comments
Labels
backlog enhancement New feature or request

Comments

@krokoko
Copy link
Collaborator

krokoko commented Mar 11, 2024

Describe the feature

In some cases, the user needs to provide custom logic to deploy a HF model to a SG endpoint:

  • overwrite/add dependencies to the existing DLC
  • add custom inference script to manage inference requests
    In those cases, the user needs to use a notebook instance, SG Studio or a local environment to
    This step could be managed by the construct

Use Case

Avoid context switching and deploy more models directly through CDK

Proposed Solution

Add a new optional property customCode pointing to a location (local or S3) containing the custom logic of the user. If provided, the construct would run a CodeBuild step to:

  • pull model artifacts from HF to an S3 location
  • add user's custom logic
  • deploy the model from the new S3 location

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change
@krokoko krokoko added the needs-triage This issue or PR still needs to be triaged. label Mar 11, 2024
@krokoko krokoko added backlog and removed needs-triage This issue or PR still needs to be triaged. labels Mar 18, 2024
@krokoko krokoko added the enhancement New feature or request label Aug 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backlog enhancement New feature or request
Projects
Development

No branches or pull requests

1 participant