Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-creating materialized view fails (gcp.bigquery.Table) #2491

Closed
brent-statsig opened this issue Oct 1, 2024 · 3 comments · Fixed by #2498
Closed

Re-creating materialized view fails (gcp.bigquery.Table) #2491

brent-statsig opened this issue Oct 1, 2024 · 3 comments · Fixed by #2498
Assignees
Labels
kind/bug Some behavior is incorrect or out of spec resolution/fixed This issue was fixed

Comments

@brent-statsig
Copy link

brent-statsig commented Oct 1, 2024

Describe what happened

package.json

"dependencies": {
    "@pulumi/gcp": "^8.3.1",
    "@pulumi/pulumi": "^3.112.0",
  }

Updating a materialize view in bigquery triggers a replacement, that fails, because the table already exists in bigquery.

Sample program

  • Create a bigquery table (gcp.bigquery.Table resource) with a materialized view
  • pulumi up
  • Update the SQL defining the materialized view
  • error

Log output

error: 1 error occurred:
    	* googleapi: Error 409: Already Exists: Table my-project:my-source.my-materialized-table, duplicate
    	*

Affected Resource(s)

No response

Output of pulumi about

pulumi about
Enter your passphrase to unlock config/secrets
    (set PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE to remember):
Enter your passphrase to unlock config/secrets
CLI
Version      3.119.0
Go Version   go1.22.4
Go Compiler  gc

Plugins
KIND      NAME    VERSION
resource  gcp     8.3.1
resource  gcp     8.2.0
resource  gcp     7.18.0
language  nodejs  unknown

Host
OS       darwin
Version  13.0
Arch     arm64

This project is written in nodejs: executable='/Users/brentechols/.nvm/versions/node/v20.11.1/bin/node' version='v20.11.1'

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

@brent-statsig brent-statsig added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Oct 1, 2024
@brent-statsig brent-statsig changed the title Re-creating materialized view fails Re-creating materialized view fails (gcp.bigquery.Table) Oct 1, 2024
@brent-statsig
Copy link
Author

Oh - I figured out the issue, deleteBeforeReplace needs to be enabled. This feels very unintuitive though, especially since so many GCP resources are globally unique by resource id, and will run into this problem. Especially with things like tables/datasets/etc, which will be referenced by their name external to pulumi, there is no "uptime" friendly way to do a replace.

Can these resources be changed to have deleteBeforeReplace be defaulted to true?

@VenelinMartinov
Copy link
Contributor

Hey @brent-statsig, thanks for reporting and sorry you've hit this.

Glad you found a workaround, that seems like the right way to go about it. Unfortunately, you've hit a bit of a corner case with the pulumi resource model: pulumi/pulumi#918

I'll look into adding the deleteBeforeReplace by default into the provider as that does help with most of these cases.

@VenelinMartinov VenelinMartinov removed the needs-triage Needs attention from the triage team label Oct 2, 2024
@VenelinMartinov VenelinMartinov self-assigned this Oct 3, 2024
VenelinMartinov added a commit that referenced this issue Oct 3, 2024
Bigquery materialized views on tables seem to be structural, so due to
pulumi/pulumi#918 the normal pulumi replace of
creating and then deleting does not work.

This PR enables DeleteBeforeRelace by default for the resource, fixing
the issue.

fixes #2491
@pulumi-bot pulumi-bot added the resolution/fixed This issue was fixed label Oct 3, 2024
@pulumi-bot
Copy link
Contributor

This issue has been addressed in PR #2498 and shipped in release v8.5.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Some behavior is incorrect or out of spec resolution/fixed This issue was fixed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants