Add upstream installation docs (#619)

This change adds quite a few edit rules to the upstream installation
docs:
- skip Namespaces section because the examples are not helpful for
Pulumi
- strip any caveats about Secrets, since Pulumi does encrypt them
- rewrite credential usage section to be more Pulumi specific and less
confusing

Since the rewrites are somewhat extensive, this particular
implementation adds a hard error during registrydocsgen for when the
upstream documentation has changed, so that we are made aware of any
changes.


- **Add automation**
- **Add installation docs for Vault from upstream**

.ci-mgmt.yaml

@@ -26,3 +26,4 @@ actions:
       run: |
         cd provider && go test -v -json -count=1 -cover -timeout 2h -tags=${{ matrix.language }} -parallel 4 . 2>&1 | tee /tmp/gotest.log | gotestfmt
 pulumiConvert: 1
+registryDocs: true

Makefile

@@ -29,7 +29,7 @@ development: install_plugins provider build_sdks install_sdks
 
 build: install_plugins provider build_sdks install_sdks
 
-build_sdks: build_nodejs build_python build_dotnet build_go build_java 
+build_sdks: build_nodejs build_python build_dotnet build_go build_java build_registry_docs
 
 install_go_sdk:
 
@@ -96,6 +96,10 @@ build_python: upstream
 		cd ./bin && \
 		../venv/bin/python -m build .
 
+# Run the bridge's registry-docs command to generated the content of the installation docs/ folder at provider repo root
+build_registry_docs:
+	$(WORKING_DIR)/bin/$(TFGEN) registry-docs --out $(WORKING_DIR)/docs
+
 clean:
 	rm -rf sdk/{dotnet,nodejs,go,python}
 

provider/installation-replaces/overview-desired.md

@@ -0,0 +1,2 @@
+The Vault provider allows Terraform to read from, write to, and configure
+[HashiCorp Vault](https://vaultproject.io/).

provider/installation-replaces/overview-input.md

@@ -0,0 +1,17 @@
+The Vault provider allows Terraform to read from, write to, and configure
+[HashiCorp Vault](https://vaultproject.io/).
+
+~> **Important** Interacting with Vault from Terraform causes any secrets
+that you read and write to be persisted in both Terraform's state file
+*and* in any generated plan files. For any Terraform module that reads or
+writes Vault secrets, these files should be treated as sensitive and
+protected accordingly.
+
+This provider serves two pretty-distinct use-cases, which each have their
+own security trade-offs and caveats that are covered in the sections that
+follow. Consider these carefully before using this provider within your
+Terraform configuration.
+
+-> Visit the [Inject secrets into Terraform using the Vault provider](https://learn.hashicorp.com/tutorials/terraform/secrets-vault?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS) Learn tutorial to learn how to use
+short-lived credentials from Vault's AWS Secrets Engine to authenticate the
+AWS provider.

provider/installation-replaces/using-credentials-desired.md

@@ -0,0 +1,25 @@
+~> **Important** It is important to ensure that the Vault token
+has a long enough `time-to-live` to allow for all Vault resources to
+be successfully provisioned. In the case where the `TTL` is insufficient,
+you may encounter unexpected permission denied errors.
+See [Vault Token TTLs](https://vaultproject.io/docs/concepts/tokens#token-time-to-live-periodic-tokens-and-explicit-max-ttls)
+for more details.
+
+Most Pulumi providers require credentials to interact with a third-party
+service that they wrap. This provider allows such credentials to be obtained
+from Vault, which means that operators or systems running Pulumi need
+only access to a suitably-privileged Vault token in order to temporarily
+lease the credentials for other providers.
+
+To reduce the exposure of secrets, the provider requests a Vault token
+with a relatively-short TTL (20 minutes, by default) which in turn means
+that where possible Vault will revoke any issued credentials after that
+time, but in particular it is unable to retract any static secrets such as
+those stored in Vault's "generic" secret backend.
+
+The requested token TTL can be controlled by the `max_lease_ttl_seconds`
+provider argument described below.
+
+Except as otherwise noted, the resources that read secrets from Vault
+are designed such that they require only the *read* capability on the relevant
+resources.

provider/installation-replaces/using-credentials-input.md

@@ -0,0 +1,35 @@
+~> **Important** It is important to ensure that the Vault token
+has a long enough `time-to-live` to allow for all Vault resources to
+be successfully provisioned. In the case where the `TTL` is insufficient,
+you may encounter unexpected permission denied errors.
+See [Vault Token TTLs](https://developer.hashicorp.com/vault/docs/concepts/tokens#token-time-to-live-periodic-tokens-and-explicit-max-ttls)
+for more details.
+
+Most Terraform providers require credentials to interact with a third-party
+service that they wrap. This provider allows such credentials to be obtained
+from Vault, which means that operators or systems running Terraform need
+only access to a suitably-privileged Vault token in order to temporarily
+lease the credentials for other providers.
+
+Currently, Terraform has no mechanism to redact or protect secrets that
+are returned via data sources, so secrets read via this provider will be
+persisted into the Terraform state, into any plan files, and in some cases
+in the console output produced while planning and applying. These artifacts
+must therefore all be protected accordingly.
+
+To reduce the exposure of such secrets, the provider requests a Vault token
+with a relatively-short TTL (20 minutes, by default) which in turn means
+that where possible Vault will revoke any issued credentials after that
+time, but in particular it is unable to retract any static secrets such as
+those stored in Vault's "generic" secret backend.
+
+The requested token TTL can be controlled by the `max_lease_ttl_seconds`
+provider argument described below. It is important to consider that Terraform
+reads from data sources during the `plan` phase and writes the result into
+the plan. Thus, a subsequent `apply` will likely fail if it is run after the
+intermediate token has expired, due to the revocation of the secrets that
+are stored in the plan.
+
+Except as otherwise noted, the resources that read secrets from Vault
+are designed such that they require only the *read* capability on the relevant
+resources.

provider/resources.go

@@ -15,7 +15,9 @@
 package provider
 
 import (
+	"bytes"
 	"fmt"
+	"os"
 	"path/filepath"
 	"strings"
 	"unicode"
@@ -28,6 +30,7 @@ import (
 
 	"github.com/pulumi/pulumi-terraform-bridge/v3/pkg/tfbridge"
 	tks "github.com/pulumi/pulumi-terraform-bridge/v3/pkg/tfbridge/tokens"
+	"github.com/pulumi/pulumi-terraform-bridge/v3/pkg/tfgen"
 	shimv2 "github.com/pulumi/pulumi-terraform-bridge/v3/pkg/tfshim/sdk-v2"
 	"github.com/pulumi/pulumi/sdk/v3/go/common/tokens"
 
@@ -547,7 +550,20 @@ func Provider() tfbridge.ProviderInfo {
 }
 
 func docEditRules(defaults []tfbridge.DocsEdit) []tfbridge.DocsEdit {
-	return append(defaults, oktaAuthBackedUserImport)
+	edits := []tfbridge.DocsEdit{
+		// These sections would trigger other edit rules so they must run first for discovery.
+		cleanUpSecretsWarnings,
+	}
+	edits = append(edits,
+		defaults...,
+	)
+	return append(edits,
+		oktaAuthBackedUserImport,
+		skipConfiguringAndPopulatingSection,
+		skipBestPracticesSection,
+		skipTutorialsSection,
+		skipNamespacesSection,
+	)
 }
 
 var oktaAuthBackedUserImport = tfbridge.DocsEdit{
@@ -563,3 +579,80 @@ var oktaAuthBackedUserImport = tfbridge.DocsEdit{
 }
 
 var missingDocs = &tfbridge.DocInfo{AllowMissing: true}
+
+// Pulumi does encrypt secrets, so redact TF specific warnings.
+var cleanUpSecretsWarnings = tfbridge.DocsEdit{
+	Path: "index.html.markdown",
+	Edit: func(_ string, content []byte) ([]byte, error) {
+		files := []string{
+			"using-credentials",
+			"overview",
+		}
+
+		replacesDir := "provider/installation-replaces/"
+		for _, file := range files {
+			input, err := os.ReadFile(replacesDir + file + "-input.md")
+			if err != nil {
+				return nil, err
+			}
+			replace, err := os.ReadFile(replacesDir + file + "-desired.md")
+			if err != nil {
+				return nil, err
+			}
+			if bytes.Contains(content, input) {
+				content = bytes.ReplaceAll(
+					content,
+					input,
+					replace)
+			} else {
+				// Hard error to ensure we keep this content up to date
+				return nil, fmt.Errorf("could not find text in upstream index.html.markdown, "+
+					"verify file content at %s", replacesDir+file+"-input.md")
+			}
+
+		}
+		return content, nil
+	},
+}
+
+// Removes a "Best Practices" section that includes TF-specific recommendations
+var skipBestPracticesSection = tfbridge.DocsEdit{
+	Path: "index.html.markdown",
+	Edit: func(_ string, content []byte) ([]byte, error) {
+		return tfgen.SkipSectionByHeaderContent(content, func(headerText string) bool {
+			return headerText == "Best Practices"
+		})
+	},
+}
+
+// Removes a "Configuring and Populating Vault" section that talks about secrets only
+var skipConfiguringAndPopulatingSection = tfbridge.DocsEdit{
+	Path: "index.html.markdown",
+	Edit: func(_ string, content []byte) ([]byte, error) {
+		return tfgen.SkipSectionByHeaderContent(content, func(headerText string) bool {
+			return headerText == "Configuring and Populating Vault"
+		})
+	},
+}
+
+// This part of the documentation is very TF-specific.
+// We should consider creating our own tutorial if the need arises.
+// See https://github.com/pulumi/pulumi-vault/issues/618.
+var skipNamespacesSection = tfbridge.DocsEdit{
+	Path: "index.html.markdown",
+	Edit: func(_ string, content []byte) ([]byte, error) {
+		return tfgen.SkipSectionByHeaderContent(content, func(headerText string) bool {
+			return headerText == "Namespace support"
+		})
+	},
+}
+
+// Removes a TF-specific "Tutorials" section
+var skipTutorialsSection = tfbridge.DocsEdit{
+	Path: "index.html.markdown",
+	Edit: func(_ string, content []byte) ([]byte, error) {
+		return tfgen.SkipSectionByHeaderContent(content, func(headerText string) bool {
+			return headerText == "Tutorials"
+		})
+	},
+}