Skip to content

Commit

Permalink
Update Jmespath shape traversal codegen to support multi-select lists…
Browse files Browse the repository at this point in the history
… following projection expressions (#3987)

## Motivation and Context
Adds support for JMESPath multi-select lists following projection
expressions such as `lists.structs[].[optionalInt, requiredInt]` (list
projection followed by multi-select lists), `lists.structs[<some filter
condition>].[optionalInt, requiredInt]` (filter projection followed by
multi-select lists), and `maps.*.[optionalInt, requiredInt]` (object
projection followed by multi-select lists).

## Description
This PR adds support for the said functionality. Prior to the PR, the
expressions above ended up the codegen either failing to generate code
(for list projection) or generating the incorrect Rust code (for filter
& object projection).

All the code changes except for `RustJmespathShapeTraversalGenerator.kt`
are primarily adjusting the existing code based on the updates made to
`RustJmespathShapeTraversalGenerator.kt`.

The gist of the code changes in `RustJmespathShapeTraversalGenerator.kt`
is as follows:
- `generateProjection` now supports `MultiSelectListExpression` on the
right-hand side (RHS).
- Previously, given `MultiSelectListExpression` on RHS, the `map`
function applied to the result of the left-hand side of a projection
expression (regardless of whether it's list, filter, or object
projection) returned a type `Option<Vec<&T>>`, and the `map` function
body used the `?` operator to return early as soon as it encountered a
field value that was `None`. That did not yield the desired behavior.
Given the snippet `lists.structs[].[optionalInt, requiredInt]` in the
`Motivation and Context` above for instance, the `map` function used to
look like this:
```
fn map(_v: &crate::types::Struct) -> Option<Vec<&i32>> {
    let _fld_1 = _v.optional_int.as_ref()?;
    let _fld_2 = _v.required_int;
    let _msl = vec![_fld_1, _fld_2];
    Some(_msl)
```
This meant if the `optional_int` in a `Struct` was `None`, we lost the
chance to access the `required_int` field when we should've. Instead,
the `map` function now looks like:
```
fn map(_v: &crate::types::Struct) -> Option<Vec<Option<&i32>>> {
    let _fld_1 = _v.optional_int.as_ref();
    let _fld_2 = Some(_v.required_int);
    let _msl = vec![_fld_1, _fld_2];
    Some(_msl)
```
This way, the `map` function body has a chance to access all the fields
of `Struct` even when any of the optional fields in a `Struct` is
`None`.
- Given the update to the signature of the `map` function above,
`generate*Projection` functions have adjusted their implementations
(such as [preserving the output type of the whole projection
expression](https://github.com/smithy-lang/smithy-rs/blob/01fed784d5fc2ef6743a336496d91a51f01e6ab2/codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/generators/waiters/RustJmespathShapeTraversalGenerator.kt#L989-L1010)
and performing [additional flattening for
`Option`s](https://github.com/smithy-lang/smithy-rs/blob/01fed784d5fc2ef6743a336496d91a51f01e6ab2/codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/generators/waiters/RustJmespathShapeTraversalGenerator.kt#L757-L760)).
 
Note that the output type of the whole projection expression stays the
same before and after this PR; it's just that the inner `map` function
used by the projection expression has been tweaked.

## Testing
- Confirmed existing tests continued working (CI and release pipeline).
- Added additional JMESPath codegen tests in
`RustJmespathShapeTraversalGeneratorTest.kt`.
- Confirmed that the updated service model with a JMESPath expression
like `Items[*].[A.Name, B.Name, C.Name, D.Name][]` generated the
expected Rust code and behaved as expected.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
  • Loading branch information
ysaito1001 authored Jan 28, 2025
1 parent b36447e commit dae6b26
Show file tree
Hide file tree
Showing 9 changed files with 350 additions and 121 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ async fn assert_error_not_transient(error: ReplayedEvent) {
let client = Client::from_conf(config);
let _item = client
.get_item()
.table_name("arn:aws:dynamodb:us-east-2:333333333333:table/table_name")
.key("foo", AttributeValue::Bool(true))
.send()
.await
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import software.amazon.smithy.rust.codegen.client.smithy.generators.config.confi
import software.amazon.smithy.rust.codegen.client.smithy.generators.config.loadFromConfigBag
import software.amazon.smithy.rust.codegen.client.smithy.generators.waiters.RustJmespathShapeTraversalGenerator
import software.amazon.smithy.rust.codegen.client.smithy.generators.waiters.TraversalBinding
import software.amazon.smithy.rust.codegen.client.smithy.generators.waiters.TraversalContext
import software.amazon.smithy.rust.codegen.client.smithy.generators.waiters.TraversedShape
import software.amazon.smithy.rust.codegen.core.rustlang.CargoDependency
import software.amazon.smithy.rust.codegen.core.rustlang.RustType
Expand Down Expand Up @@ -168,16 +169,18 @@ class EndpointParamsInterceptorGenerator(
TraversedShape.from(model, operationShape.inputShape(model)),
),
),
TraversalContext(retainOption = false),
)

when (pathTraversal.outputType) {
is RustType.Vec -> {
rust(".$setterName($getterName(_input))")
}

else -> {
rust(".$setterName($getterName(_input).cloned())")
if (pathTraversal.outputType.member is RustType.Reference) {
rust(".$setterName($getterName(_input).map(|v| v.into_iter().cloned().collect::<Vec<_>>()))")
} else {
rust(".$setterName($getterName(_input))")
}
}
else -> rust(".$setterName($getterName(_input).cloned())")
}
}

Expand Down Expand Up @@ -211,6 +214,7 @@ class EndpointParamsInterceptorGenerator(
TraversedShape.from(model, operationShape.inputShape(model)),
),
),
TraversalContext(retainOption = false),
)

rust("// Generated from JMESPath Expression: $pathValue")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,8 @@ internal class EndpointResolverGenerator(
"clippy::comparison_to_empty",
// we generate `if let Some(_) = ... { ... }`
"clippy::redundant_pattern_matching",
// we generate `if (s.as_ref() as &str) == ("arn:") { ... }`, and `s` can be either `String` or `&str`
"clippy::useless_asref",
)
private val context = Context(registry, runtimeConfig)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ import software.amazon.smithy.rulesengine.traits.ExpectedEndpoint
import software.amazon.smithy.rust.codegen.client.smithy.ClientCodegenContext
import software.amazon.smithy.rust.codegen.client.smithy.endpoint.Types
import software.amazon.smithy.rust.codegen.client.smithy.endpoint.rustName
import software.amazon.smithy.rust.codegen.client.smithy.generators.ClientInstantiator
import software.amazon.smithy.rust.codegen.core.rustlang.Writable
import software.amazon.smithy.rust.codegen.core.rustlang.docs
import software.amazon.smithy.rust.codegen.core.rustlang.escape
Expand Down Expand Up @@ -50,8 +49,6 @@ internal class EndpointTestGenerator(
"capture_request" to RuntimeType.captureRequest(runtimeConfig),
)

private val instantiator = ClientInstantiator(codegenContext)

private fun EndpointTestCase.docs(): Writable {
val self = this
return writable { docs(self.documentation.orElse("no docs")) }
Expand Down Expand Up @@ -134,7 +131,23 @@ internal class EndpointTestGenerator(
value.values.map { member ->
writable {
rustTemplate(
"#{Document}::from(#{value:W})",
/*
* If we wrote "#{Document}::from(#{value:W})" here, we could encounter a
* compile error due to the following type mismatch:
* the trait `From<Vec<Document>>` is not implemented for `Vec<std::string::String>`
*
* given the following method signature:
* fn resource_arn_list(mut self, value: impl Into<::std::vec::Vec<::std::string::String>>)
*
* with a call site like this:
* .resource_arn_list(vec![::aws_smithy_types::Document::from(
* "arn:aws:dynamodb:us-east-1:333333333333:table/table_name".to_string(),
* )])
*
* For this reason we use `into()` instead to allow types that need to be converted
* to `Document` to continue working as before, and to support the above use case.
*/
"#{value:W}.into()",
*codegenScope,
"value" to generateValue(member),
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ package software.amazon.smithy.rust.codegen.client.smithy.endpoint.rulesgen
import org.jetbrains.annotations.Contract
import software.amazon.smithy.rulesengine.language.evaluation.type.BooleanType
import software.amazon.smithy.rulesengine.language.evaluation.type.OptionalType
import software.amazon.smithy.rulesengine.language.evaluation.type.StringType
import software.amazon.smithy.rulesengine.language.evaluation.type.Type
import software.amazon.smithy.rulesengine.language.syntax.expressions.Expression
import software.amazon.smithy.rulesengine.language.syntax.expressions.ExpressionVisitor
Expand All @@ -24,6 +25,7 @@ import software.amazon.smithy.rust.codegen.core.rustlang.rust
import software.amazon.smithy.rust.codegen.core.rustlang.rustTemplate
import software.amazon.smithy.rust.codegen.core.rustlang.writable
import software.amazon.smithy.rust.codegen.core.util.PANIC
import java.lang.RuntimeException

/**
* Root expression generator.
Expand Down Expand Up @@ -56,7 +58,18 @@ class ExpressionGenerator(
else -> rust("${ref.name.rustName()}.to_owned()")
}
} else {
rust(ref.name.rustName())
try {
when (ref.type()) {
// This ensures we obtain a `&str`, regardless of whether `ref.name.rustName()` returns a `String` or a `&str`.
// Typically, we don't know which type will be returned due to code generation.
is StringType -> rust("${ref.name.rustName()}.as_ref() as &str")
else -> rust(ref.name.rustName())
}
} catch (_: RuntimeException) {
// Because Typechecking was never invoked upon calling `.type()` on Reference for an expression
// like "{ref}: rust". See `generateLiterals2` in ExprGeneratorTest.
rust(ref.name.rustName())
}
}
}

Expand Down
Loading

0 comments on commit dae6b26

Please sign in to comment.