AWS S3 SDK Migration from V1 to V2

[elmiomar]

This PR migrates all components of the distribution service that rely on AWS S3 from AWS SDK for Java V1 to V2, addressing the upcoming end-of-support for SDK V1, scheduled for December 31, 2025 (AWS Announcement). This has all the necessary code and test changes.

Motivation

1. AWS SDK V1 End-of-Support:

• We are currently using the following SDK V1 dependencies:

  • aws-java-sdk-core: version 1.12.100 (November 3, 2021)
  • aws-java-sdk-s3: version 1.12.261 (July 15, 2022)

• Migration to SDK V2 ensures continued support, and better performance.

2. Performance Optimization:

• In SDK V1, a draining logic was required to reuse HTTP connections, which, if missed, could lead to OutOfMemoryError or connection exhaustion.
• SDK V2 eliminates this need by efficiently managing HTTP connections. Simply closing the ResponseInputStream releases the connection back to the pool. (source)

Important Changes

1. Code Migration:

• All S3 operations now use AWS SDK for Java V2 (version 2.29.35):

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>aws-sdk-java</artifactId>
    <version>2.29.35</version>
</dependency>

• Removed the draining logic required in SDK V1.

2. Local MD5 Checksum Validation:

• Since S3Mock does not validate the contentMD5 field during tests, local validation was added in the saveAs() method.
• Real AWS S3 Behavior:
  • AWS S3 validates the checksum and throws an InvalidDigest error if mismatched.
• Testing Context:
  • Local validation ensures data integrity during unit tests.
  • In a production environment, AWS S3 performs its own checksum validation.

3. Property Update:

• Added cloud.aws.region=us-east-1 to @TestPropertySource for tests involving controllers.
  This resolves region-related errors during tests, as SDK V2 requires an explicitly defined region.

4. Unit Test Updates:

• Adjusted unit tests to match SDK V2 logic

5. New AWSS3TemplateTest:

• Created a new AWSS3TemplateTest file that serves as a template for AWS S3 unit tests using a mock server. This file provides a skeleton to easily create S3 unit tests.

Testing

• All unit tests pass.
• oar-docker testing in progress...