Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSDK-9651 - Add cloud metadata and api key to module env vars #4736

Merged
merged 5 commits into from
Jan 24, 2025

Conversation

cheukt
Copy link
Member

@cheukt cheukt commented Jan 23, 2025

added them in the processConfig step so that changes to API keys can cause a module reconfigure. otherwise API keys may change without changing the actual environment the module is running in.

In theory, we can add the extra env vars later in the process (around when we create the NewModuleManager), but opted not to because of then env vars and the actual api key might get out of sync

example log for a cloud robot

2025-01-23T15:43:55.936Z        INFO    rdk.modmanager.rand_fake-modules-go       utils/env.go:140        Starting module with following Viam environment variables {"environment":["VIAM_MODULE_ID=rand:fake-modules-go","VIAM_API_KEY=XXXXXXXXXX","VIAM_PRIMARY_ORG_ID=e76d1b3b-0468-4efd-bb7f-fb1d2b352fcb","VIAM_MACHINE_ID=500c0986-a76e-44c1-97ac-aa17bc175b37","VIAM_MODULE_ROOT=/Users/cheukt/.viam/packages/data/module/235a3cfa-115d-445e-a165-05cbd3dbc615-fake-modules-go-0_0_12-darwin-arm64","VIAM_HOME=/Users/cheukt/.viam","VIAM_MODULE_DATA=/Users/cheukt/.viam/module-data/7ecca299-c025-4d35-bc49-46b38ac416f3/rand_fake-modules-go","VIAM_LOCATION_ID=hpwe3f5esh","VIAM_MACHINE_PART_ID=7ecca299-c025-4d35-bc49-46b38ac416f3","VIAM_API_KEY_ID=138475a6-3aaa-41bc-8833-41a803aaf9df"]}

example log for a local

2025-01-23T15:46:07.225Z        INFO    rdk.modmanager.SimpleModule     utils/env.go:140        Starting module with following Viam environment variables  {"environment":["VIAM_HOME=/Users/cheukt/.viam","VIAM_MODULE_DATA=/Users/cheukt/.viam/module-data/local/SimpleModule"]}

@cheukt cheukt requested a review from benjirewis January 23, 2025 15:45
@viambot viambot added the safe to test This pull request is marked safe to test from a trusted zone label Jan 23, 2025
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 23, 2025
Copy link
Member

@benjirewis benjirewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! Just some questions.

config/module.go Outdated Show resolved Hide resolved
@@ -596,6 +627,13 @@ func processConfig(unprocessedConfig *Config, fromCloud bool, logger logging.Log
}
}

// add additional environment vars to modules
// adding them here ensures that if the parsed API key changes, the module will be restarted with the updated environment.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the module will be restarted

Where is that happening?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this happens when we diff the config (the env being different on the module config will cause a reconfigure on the module), so more of just utilizing the existing mechanisms to make sure the module is running in the most up to date environment

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool that makes sense; thanks.

continue
}
// the keys come in unsorted, so sort the keys so we'll always get the same API key
// if there are no changes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we know that the first API key is the one the module wants?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't - it was mentioned during 6 week check-in that we should just make it easy to access, so opted for this instead of returning all of them.

env = append(env, v)
}
for key, val := range envVars {
// mask the secret
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will the module still get access to this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ya, the map passed in is not updated

Co-authored-by: Benjamin Rewis <[email protected]>
@cheukt cheukt requested a review from benjirewis January 23, 2025 22:21
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 23, 2025
Copy link
Member

@benjirewis benjirewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; thanks.

@@ -30,6 +31,31 @@ const (

// AndroidFilesDir is hardcoded because golang inits before our android code can override HOME var.
AndroidFilesDir = "/data/user/0/com.viam.rdk.fgservice/cache"

// ViamEnvVarPrefix is the prefix for all Viam-related environment variables.
ViamEnvVarPrefix = "VIAM_"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah new env vars should definitely start with VIAM_ now; I feel like that was already an expectation, but thanks for encoding more formally.

@cheukt cheukt merged commit 1e62e54 into main Jan 24, 2025
16 checks passed
@cheukt cheukt deleted the add-cloud-metadata-to-env-vars branch January 24, 2025 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
safe to test This pull request is marked safe to test from a trusted zone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants