Unable to unmarshall exception response with the unmarshallers provided

[alibk0rd]

Describe the bug

After upgrading to Java 17, Amazon Java (DynamoDB) SDK fails to unmarshall some errors:

com.amazonaws.AmazonServiceException: Unable to unmarshall exception response with the unmarshallers provided (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: K7IOB4TL27I8KG12J3OJ28JGAFVV4KQNSO5AEMVJF66Q9ASUAAJG; Proxy: null)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:6200)
	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:6167)
	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeUpdateItem(AmazonDynamoDBClient.java:5785)
	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.updateItem(AmazonDynamoDBClient.java:5749)
	at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper$SaveObjectHandler.doUpdateItem(DynamoDBMapper.java:871)
	at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper$2.executeLowLevelRequest(DynamoDBMapper.java:611)
	at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper$SaveObjectHandler.execute(DynamoDBMapper.java:750)
	at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper.save(DynamoDBMapper.java:640)
	at com.amazonaws.services.dynamodbv2.datamodeling.AbstractDynamoDBMapper.save(AbstractDynamoDBMapper.java:128)
...

This is masking the real issue/error by failing to unmarshall the details.

Expected Behavior

To have the real/underlying error details in the logs

Current Behavior

Fails to unmarshall the error details, masking the real underlying issue or error

Reproduction Steps

In at least one case the underlying error that the SDK fails to unmarshall is:

com.amazonaws.services.dynamodbv2.model.ResourceInUseException: Cannot create preexisting table (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ResourceInUseException; Request ID: 17cbf84f-e818-499d-9cf7-5c636f420282; Proxy: null)

The underlying issue is reproduced by trying to recreate an existing DynamoDB table, but the end result is instead an exception of type AmazonServiceException with the error message:

Unable to unmarshall exception response with the unmarshallers provided

Masking the root cause.

Possible Solution

The upgrade to SDK v2 as suggested elsewhere is not a viable solution for us as AWS SDK v2 is not supported by Spring AWS framework yet.
Also, DynamoDB SDK v2 is yet to have feature parity with v1 (specifically DynamoDBMapper).

Additional Information/Context

No response

AWS Java SDK version used

1.12.243

JDK version used

17.0.3

Operating System and version

Linux (Arch) Kernel: 5.18.7-arch1-1

[debora-ito]

Thank you for reaching out @alibk0rd.

We'll investigate the root cause of the unmarshalling error and better define the expectation of using JDK 17 with SDK v1. We'll update the README and other related docs with the results of the investigation.

[hsteidel]

Ran into this today.

JDK 17.03
Spring Boot: 2.6.9
Spring-Integration-AWS: 2.5.1
aws-java-sdk-dynamodb: 1.12.260
Jackson Databind: 2.13.3

I turned on log level debug and the root cause seems to be

 Unable to make final void java.lang.Throwable.setCause(java.lang.Throwable) accessible: module java.base does not "opens java.lang" to unnamed module @4b0b0854

I might be wrong, but what I read according to the link below, is that this issue is slated to be fixed in Jackson Databind 2.13.4/2.14.0.
FasterXML/jackson-databind#3275

I tried downgrading a few dependencies as a workaround and no luck. I understand there are a few illegal-access jvm flags that can potentially be used in some Java versions, but I don't want to go there.

Of note, this only happens to me when I set the setTTL option on the Table's config. Otherwise, I don't get this and my app works just fine. The offending call seems to be:

Sending Request: POST https://dynamodb.us-east-2.amazonaws.com / Headers: (amz-sdk-invocation-id: 0da40f4c-e922-e913-3bed-153132415507, Content-Length: 102, Content-Type: application/x-amz-json-1.0, User-Agent: aws-sdk-java/1.12.260 Mac_OS_X/12.4 OpenJDK_64-Bit_Server_VM/17.0.3+6-LTS java/17.0.3 vendor/Amazon.com_Inc. cfg/retry-mode/legacy, X-Amz-Target: DynamoDB_20120810.UpdateTimeToLive, ) 

Update for anyone else who is running into this for Spring Integration ConcurrentMetadataStore purposes, needs a quick solution, and wants to future proof. It's not too bad of a job to implement your own ConcurrentMetadataStore using the Java SDK V2 DynamoDbClient, DynamoDbEnhancedClient and DynamoDbTable<Bean> APIs. I found it actually simplifies the implementation.

edit: Got a few private requests for sharing an example. This is just a starter example to get you going.

https://gist.github.com/hsteidel/16446c7df65abeb95e44fd38e4f4dd62

[debora-ito]

Hello everyone,

after some deliberation we decided to clarify the scope of JDK support for the Java SDK v1. We added a section to the README:
https://github.com/aws/aws-sdk-java#maintenance-and-support-for-java-versions

Essentially, the Java SDK v1 will support Java versions from 7 to 16.

If you are not able to migrate to Java SDK v2, the README section provides some workarounds based on the error you're experiencing. Please let us know of any error messages that are not on that list, we'll then provide a workaround too.

[github-actions]

It looks like this issue has not been active for more than five days. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.

[storytime]

Same issue.

Pushing object

Error: