How to pull secrets that are part of a json structure?

[joannbrereton]

So I've moved along with this and now I've run into a second problem.

I've entered a secret with key and value into AWS Secrets Manager
e.g.

Entered key "my/secret/name" as "my/secret/key" and "my/secret/value" as shown

So, I've entered the following into my secrets.yml:

MY_SECRET: !var my/secret/key/name

If I use

summon -p summon-aws-secrets env

What I get returned is:

MY_SECRET={"mysecretkey":"mysecretvalue"}

which is unfortunately, not very useful if I want to pass it to, for instance, my docker-compose up command which actually expects MY_SECRET=mysecretvalue.

Is there some notation that let's me pull the value associated with "mysecretkey" inside of the AWS secret named my/secret/name?

[jepperson2]

@joannbrereton I'm not sure if what you're talking about is possible, but one workaround is to simply store mysecretvalue as Plaintext, erasing mysecretkey altogether

If you click on the Plaintext tab and erase everything but the secretvalue, Summon will fetch it in the format you want.

I understand that this may not be desirable, but you are keying on the secret name already.

[doodlesbykumbi]

Yeah. It seems that AWS secrets can be ASCII letters, digits, or any of the following characters: /_+=.@-. We can standardise on JSON key accessor by introducing a separator. Not sure what that character would be though ...

[joannbrereton]

@jepperson2 OK, I'll see what I can do about that. It's part of a particular solution so I'm not sure I'll be able to move it to plain text but I'll check around to see if that would break anything else.

[jepperson2]

@joannbrereton Alternatively, you could wrap whatever consumes that secret in some parsing logic, or even run it through jq if you have a lot of key value pairs/nests.

[doodlesbykumbi]

Here's a proposal:

Variable IDs can take the form my/secret/key/name#mysecretkey
The suffix #mysecretkey is optional and only relevant to JSON.
If the suffix is present the provider can parse the payload as JSON and grab the value under the key.

You can have the following in your secrets.yml

MY_SECRET: !var my/secret/key/name#mysecretkey

Run:

summon -p summon-aws-secrets env

Get:

MY_SECRET=mysecretvalue

PRs welcome :)

[joannbrereton]

That's pretty much what I was looking for, @doodlesbykumbi . :-) Not sure if I'll have time to PR, but I can play around when I do.

[doodlesbykumbi]

@joannbrereton try out the branch for the PR and see if it works for you :)

[sgnn7]

@joannbrereton @jepperson2 This issue should be resolved with the new v0.3.0 release. Give this new version a try and let us know if it works for you. :)

CC: @synax

Closing as resolved.

[jepperson2]

@sgnn7 This is excellent! Thank you for addressing this issue.

While the alphanumeric and /_+=.@- character restriction holds true for secret names, it does not hold true for keys or values.

As a result, keys that include '#' or '$' fail to be fetched properly. For example, a secrets.yml like this one fails:

MY_SECRET1: !var fakeSecret#$temp
MY_SECRET2: !var fakeSecret##temp
MY_SECRET3: !var fakeSecret#temp#
MY_SECRET4: !var fakeSecret#te#mp

Because MY_SECRET1 has a '$' in it, it is being parsed as though a corresponding -D temp=... should be passed to the summon command. The error I'm getting is from here: https://github.com/cyberark/summon/blob/8687c607e92d8df1206adef099db92dc1d84325c/secretsyml/secretsyml.go#L192

The command summon -D temp=\$temp -p summon-aws-secrets env succeeds in this case because it performs the substitution and then fetches the secret and parses it.

In the above example, MY_SECRET2 is populated with the entire JSON struct of the secret while MY_SECRET3 and MYSECRET4 are blank.

These are edge cases, but I'm noting them here in case users of this release run into issues with keys with these characters. One resolution to the problems with '#' could be to use strings.Index instead of strings.Split. I believe the resolution to handling the '$' character would have to be made in the Summon repo.

[doodlesbykumbi]

@jepperson2 Thank you for reporting the above.

I've recorded the 2 issues. Please have a look. I'd love feedback particularly as to whether I've characterised it well :)

1. summon empty substitution variable bug fix: substitution variable format interferes with provider key resolution summon#104
2. [RESOLVED]summon-aws-secret incorrect resolution of multiple # fix: multiple # in a secret identifier are not resolved correctly #13

[jepperson2]

@doodlesbykumbi Thank you for your prompt response and action!

They both seem well characterized.

The summon-aws-secrets resolution is nice! Using strings.SplitN was a better plan than strings.Index