Fixing Deployment Secrets - Parallel Values

[IaroslavTitov]

Note: this is an alternative approach for PR - #325

Summary

• Added logic to save cipher and plaintext separately, allowing comparison on refresh of just ciphertext, fixing Refresh on Deployment Settings always shows a diff if there are secrets #123
• Import now works as well, including code generation (with dummy values for secrets)
• Migrated to new PUT API and updated client to actually return DeploymentSettings

Testing

• Tested pulumi up, refresh, import and up from previous version of the provider (for unchanged DS inputs, migrating to this new way of saving will require refresh and then up)

Example TS program (Sadly can't use Dotnet, due to bug with maps):

const settings = new service.DeploymentSettings("deployment_settings", {
  organization: "IaroslavTitov",
  project: "PulumiDotnet",
  stack: "SdkTest5",
  operationContext: {
    environmentVariables: {
      TEST_VAR: "fooa",
      SECRET_VAR: config.requireSecret("my_secret"),
    }
  },
  sourceContext: {
    git: {
        repoUrl: "https://github.com/pulumi/deploy-demos.git",
        branch: "refs/heads/main",
        repoDir: "pulumi-programs/simple-resource",
        gitAuth: {
            sshAuth: {
                sshPrivateKey: "privatekey",
                password: secret,
            }
        }
    }
}
});

Secret resource values end up split into cipher and plaintext like this:

    "sshPrivateKeyCipher": "AAABAJrXK3j7+jgXLJI9Q30JnAFDxek6yUyYudh3h2SU4/dQKyBl",
    "sshPrivateKey": {
      "4dabf18193072939515e22adb298388d": "1b47061264138c4ac30d75fd1eb44270",
      "ciphertext": "AAABAKlREY4tsjd1bbWc8JXPaABRqG6BeVxfUkhO1ob70Q=="
    }

twin values for each secret. Environment variables ciphers are stored in a separate variable `environmentVariablesCipher`

Passwords and sshKey are forced into twin secrets, Environment Variables are optionally twin secrets, everything else uses normal Pulumi workdlows, because they are not secret in Pulumi Service.

Import of the above code generates successfully with dummy values for secrets:

const ds1 = new pulumiservice.DeploymentSettings("ds1", {
operationContext: {
environmentVariables: {
SECRET_VAR: pulumi.secret(""),
TEST_VAR: "fooa",
},
},
organization: "IaroslavTitov",
project: "PulumiDotnet",
sourceContext: {
git: {
branch: "refs/heads/main",
gitAuth: {
sshAuth: {
password: pulumi.secret(""),
sshPrivateKey: pulumi.secret(""),
},
},
repoDir: "pulumi-programs/simple-resource",
repoUrl: "https://github.com/pulumi/deploy-demos.git",
},
},
stack: "SdkTest5",
}, {
protect: true,
});

[komalali]

Doesn't this return both Value and Ciphertext?

[IaroslavTitov]

No, Pulumi Service only returns either plaintext or ciphertext, with the Secret bool indicating which one. But since we opted not to have the ability to return plaintext on these secret deployment settings, it only ever returns ciphertext.

This whole struct tree is taken from sdk, I unfrotunately had to copy it in to override the marshaling/unmarshaling logic

[IaroslavTitov]

Updated the code.
This way it's not a breaking change, which is well worth the extra bit of complexity, I just didn't see this way until today.

The env vars are now stored as map of string to object, inside which ciphertext and plaintext are packed. Import now needs a separate flag for serialization into Inputs, separate from serialization into Properties, but it's a pretty small change.

[IaroslavTitov]

Ran into an issue with python tests, filed #322 to address later, looks like another bug with providers

[komalali]

This diff is surprising in this PR. Did we forget to include this earlier?

[IaroslavTitov]

Yep, the go files only didn't generate for Environment resources, Stoo pinged me about it. Looking into it, a while ago I updated .gitignore and it's covering too much, which caused this. I'll have to address the gitignore issue, but figured should include the missed files asap

[komalali]

Can we pull this out into a separate PR and get that merged and released? I'd rather not hold that up on this PR.

[komalali]

Can we close this out @IaroslavTitov ?

[IaroslavTitov]

Closing as this was just an alternate version