chore: avoid in-place modifications of AST nodes in the partial evaluation

[i4ki]

What this PR does / why we need it:

The in-place AST modifications were a nice optimization when generation was serial but with the introduction of #1937 it's racy.

Which issue(s) this PR fixes:

none

Special notes for your reviewer:

Does this PR introduce a user-facing change?

no

[github-actions]

2024/10/24 09:15:26 Terraform detected version: 1.9.7
toolsetPath: /tmp/cmd-terramate-test-753415128
=== RUN   TestInteropCloudSyncPreview
=== RUN   TestInteropCloudSyncPreview/preview:_basic-drift
    interoperability_test.go:46: using GITHUB_EVENT_FILE=/home/runner/work/_temp/_github_workflow/event.json
=== RUN   TestInteropCloudSyncPreview/preview:_basic-drift-uppercase-id
    interoperability_test.go:46: using GITHUB_EVENT_FILE=/home/runner/work/_temp/_github_workflow/event.json
--- PASS: TestInteropCloudSyncPreview (14.42s)
    --- PASS: TestInteropCloudSyncPreview/preview:_basic-drift (7.62s)
    --- PASS: TestInteropCloudSyncPreview/preview:_basic-drift-uppercase-id (6.81s)
=== RUN   TestInteropSyncDeployment
=== RUN   TestInteropSyncDeployment/deployment:_empty
=== RUN   TestInteropSyncDeployment/deployment:_empty-uppercase-id
--- PASS: TestInteropSyncDeployment (46.29s)
    --- PASS: TestInteropSyncDeployment/deployment:_empty (23.27s)
    --- PASS: TestInteropSyncDeployment/deployment:_empty-uppercase-id (23.03s)
=== RUN   TestInteropDrift
=== RUN   TestInteropDrift/drift:_basic-drift
=== RUN   TestInteropDrift/drift:_basic-drift-uppercase-id
--- PASS: TestInteropDrift (73.98s)
    --- PASS: TestInteropDrift/drift:_basic-drift (37.03s)
    --- PASS: TestInteropDrift/drift:_basic-drift-uppercase-id (36.94s)
PASS
ok  	github.com/terramate-io/terramate/e2etests/cloud/interop	143.120s

[github-actions]

Preview of ubuntu-focal/go1.21 tests in 348133d

🔍 View Details on Terramate Cloud

generate/genhcl
hcl/ast
hcl/eval
mapexpr
stdlib

[github-actions]

Preview of macos-ventura/go1.21 tests in 348133d

🔍 View Details on Terramate Cloud

generate/genhcl
hcl/ast
hcl/eval
mapexpr
stdlib

[github-actions]

metric: time/op
ChangeDetection-4: old 4.03ms ± 4%: new 4.05ms ± 4%: delta: 0.00%
ChangeDetectionTFAndTG-4: old 12.8ms ± 2%: new 13.1ms ± 2%: delta: 2.38%
CloudReadLines-4: old 1.03ms ± 9%: new 0.98ms ± 6%: delta: -4.50%
CloudReadLine-4: old 7.25ms ± 1%: new 7.28ms ± 2%: delta: 0.00%
ListFiles-4: old 81.1µs ± 1%: new 81.7µs ± 1%: delta: 0.83%
Generate-4: old 1.57s ± 3%: new 1.60s ± 3%: delta: 1.98%
GenerateRegex-4: old 1.06s ± 6%: new 1.07s ± 4%: delta: 0.00%
TokensForExpressionComplex-4: old 1.25ms ± 0%: new 1.26ms ± 1%: delta: 0.43%
TokensForExpressionPlainStringNoNewline-4: old 794ns ± 2%: new 792ns ± 1%: delta: 0.00%
TokensForExpressionStringWith100Newlines-4: old 23.9µs ± 1%: new 24.1µs ± 2%: delta: 0.79%
TokensForExpressionObjectWith100KeysWithNumberValues-4: old 1.46ms ± 1%: new 1.45ms ± 1%: delta: -0.70%
TokensForExpression-4: old 1.27ms ± 1%: new 1.26ms ± 1%: delta: -0.50%
PartialEvalComplex-4: old 531µs ± 2%: new 544µs ± 3%: delta: 2.59%
PartialEvalSmallString-4: old 3.77µs ± 1%: new 3.95µs ± 1%: delta: 4.68%
PartialEvalHugeString-4: old 1.91ms ± 0%: new 1.92ms ± 0%: delta: 0.45%
PartialEvalHugeInterpolatedString-4: old 4.78ms ± 1%: new 4.98ms ± 1%: delta: 4.03%
PartialEvalObject-4: old 25.5µs ± 3%: new 27.1µs ± 1%: delta: 6.21%
TmAllTrueLiteralList-4: old 625µs ± 1%: new 625µs ± 2%: delta: 0.00%
TmAllTrueFuncall-4: old 20.9µs ± 1%: new 20.8µs ± 3%: delta: 0.00%
TmAnyTrueLiteralList-4: old 4.95ms ± 1%: new 4.94ms ± 2%: delta: 0.00%
TmAnyTrueFuncall-4: old 21.2µs ± 3%: new 21.2µs ± 2%: delta: 0.00%
TmTernary-4: old 2.61µs ± 2%: new 2.55µs ± 2%: delta: -2.08%
TmTryUnknownFunc-4: old 2.28µs ± 1%: new 2.30µs ± 1%: delta: 1.28%
TmTryUnknownVariable-4: old 2.18µs ± 1%: new 2.21µs ± 2%: delta: 1.47%
TmTryUnknownObjectKey-4: old 2.49µs ± 0%: new 2.55µs ± 3%: delta: 2.45%
metric: alloc/op
ChangeDetection-4: old 357kB ± 0%: new 357kB ± 0%: delta: 0.00%
ChangeDetectionTFAndTG-4: old 4.80MB ± 0%: new 4.80MB ± 0%: delta: 0.00%
CloudReadLines-4: old 3.12MB ± 0%: new 3.12MB ± 0%: delta: 0.00%
CloudReadLine-4: old 3.37MB ± 0%: new 3.37MB ± 0%: delta: 0.00%
ListFiles-4: old 27.6kB ± 0%: new 27.6kB ± 0%: delta: 0.00%
Generate-4: old 2.24GB ± 0%: new 2.24GB ± 0%: delta: 0.00%
GenerateRegex-4: old 927MB ± 0%: new 927MB ± 0%: delta: 0.00%
TokensForExpressionComplex-4: old 394kB ± 0%: new 394kB ± 0%: delta: 0.00%
TokensForExpressionPlainStringNoNewline-4: old 512B ± 0%: new 512B ± 0%: delta: 0.00%
TokensForExpressionStringWith100Newlines-4: old 12.5kB ± 0%: new 12.5kB ± 0%: delta: 0.00%
TokensForExpressionObjectWith100KeysWithNumberValues-4: old 395kB ± 0%: new 395kB ± 0%: delta: 0.00%
TokensForExpression-4: old 395kB ± 0%: new 394kB ± 0%: delta: -0.00%
PartialEvalComplex-4: old 338kB ± 0%: new 361kB ± 0%: delta: 6.69%
PartialEvalSmallString-4: old 1.74kB ± 0%: new 1.95kB ± 0%: delta: 11.93%
PartialEvalHugeString-4: old 196kB ± 0%: new 196kB ± 0%: delta: 0.11%
PartialEvalHugeInterpolatedString-4: old 4.05MB ± 0%: new 4.31MB ± 0%: delta: 6.34%
PartialEvalObject-4: old 17.2kB ± 0%: new 19.4kB ± 0%: delta: 13.34%
TmAllTrueLiteralList-4: old 321kB ± 0%: new 321kB ± 0%: delta: 0.00%
TmAllTrueFuncall-4: old 10.6kB ± 0%: new 10.6kB ± 0%: delta: 0.00%
TmAnyTrueLiteralList-4: old 2.09MB ± 0%: new 2.09MB ± 0%: delta: 0.00%
TmAnyTrueFuncall-4: old 10.7kB ± 0%: new 10.7kB ± 0%: delta: 0.00%
TmTernary-4: old 1.20kB ± 0%: new 1.18kB ± 0%: delta: -1.33%
TmTryUnknownFunc-4: old 784B ± 0%: new 784B ± 0%: delta: 0.00%
TmTryUnknownVariable-4: old 768B ± 0%: new 768B ± 0%: delta: 0.00%
TmTryUnknownObjectKey-4: old 952B ± 0%: new 952B ± 0%: delta: 0.00%
metric: allocs/op
ChangeDetection-4: old 2.43k ± 0%: new 2.43k ± 0%: delta: 0.00%
ChangeDetectionTFAndTG-4: old 33.6k ± 0%: new 33.6k ± 0%: delta: 0.00%
CloudReadLines-4: old 5.54k ± 0%: new 5.54k ± 0%: delta: 0.00%
CloudReadLine-4: old 60.0k ± 0%: new 60.0k ± 0%: delta: 0.00%
ListFiles-4: old 335 ± 0%: new 335 ± 0%: delta: 0.00%
Generate-4: old 25.7M ± 0%: new 25.7M ± 0%: delta: 0.00%
GenerateRegex-4: old 18.4M ± 0%: new 18.4M ± 0%: delta: 0.00%
TokensForExpressionComplex-4: old 4.83k ± 0%: new 4.83k ± 0%: delta: 0.00%
TokensForExpressionPlainStringNoNewline-4: old 20.0 ± 0%: new 20.0 ± 0%: delta: 0.00%
TokensForExpressionStringWith100Newlines-4: old 227 ± 0%: new 227 ± 0%: delta: 0.00%
TokensForExpressionObjectWith100KeysWithNumberValues-4: old 3.19k ± 0%: new 3.19k ± 0%: delta: 0.00%
TokensForExpression-4: old 4.83k ± 0%: new 4.83k ± 0%: delta: 0.00%
PartialEvalComplex-4: old 3.48k ± 0%: new 3.78k ± 0%: delta: 8.52%
PartialEvalSmallString-4: old 23.0 ± 0%: new 26.0 ± 0%: delta: 13.04%
PartialEvalHugeString-4: old 35.0 ± 0%: new 38.0 ± 0%: delta: 8.57%
PartialEvalHugeInterpolatedString-4: old 23.1k ± 0%: new 26.1k ± 0%: delta: 13.02%
PartialEvalObject-4: old 152 ± 0%: new 183 ± 0%: delta: 20.39%
TmAllTrueLiteralList-4: old 5.94k ± 0%: new 5.94k ± 0%: delta: 0.00%
TmAllTrueFuncall-4: old 275 ± 0%: new 275 ± 0%: delta: 0.00%
TmAnyTrueLiteralList-4: old 59.6k ± 0%: new 59.6k ± 0%: delta: 0.00%
TmAnyTrueFuncall-4: old 277 ± 0%: new 277 ± 0%: delta: 0.00%
TmTernary-4: old 28.0 ± 0%: new 27.0 ± 0%: delta: -3.57%
TmTryUnknownFunc-4: old 21.0 ± 0%: new 21.0 ± 0%: delta: 0.00%
TmTryUnknownVariable-4: old 20.0 ± 0%: new 20.0 ± 0%: delta: 0.00%
TmTryUnknownObjectKey-4: old 23.0 ± 0%: new 23.0 ± 0%: delta: 0.00%
check failed: allocs/op=+20%

[zied-elouaer]

Very nice. This branch works perfectly fine on the customer repo and take 6 seconds to run generate there!

[snakster]

👍
It could be nice have a test in the future that invokes partialEval (or something further up the call tree) in parallel to potentially catch any remaining race conditions.

The base branch was changed.