Architecture Discussion

[liyunrui]

One issue architecture in 19:28:

You make a fair observation. Based on the architecture diagram shown, it does not seem to depict the utilization of AWS's multi-variant model deployment capabilities for automatic traffic distribution across different model variants based on configured weights.

The diagram shows an Amazon SageMaker endpoint that can invoke specific model variants (v1, v2), but there is no clear indication of automatic traffic routing or weighting mechanisms employed. The architecture appears to rely on a separate "Bandit Algorithm" component to retrieve the appropriate model variant for serving requests.

Leveraging AWS's multi-variant model deployment functionality could help automate traffic distribution across different model versions/variants based on defined weights or performance metrics, without the need for a custom bandit algorithm implementation. This could potentially simplify the architecture and provide more out-of-the-box capabilities for model deployment and testing.

Overall, your observation is valid - the current architecture diagram does not seem to take full advantage of AWS's built-in multi-variant model deployment and traffic routing capabilities, which could be beneficial for managing and testing different model versions in production environments.