[RSDK-8291] Use Global Connection to App in Config Watcher #4773

bashar-515 · 2025-02-05T17:40:14Z

This PR propagates the global connection to App that's instantiated in viam-server's entrypoint down to the function actually used by the config watcher to retrieve a new config. Before, this function would create and manage its own connection to App. Now, it receives a preexisting connection as a function parameter. The changes made here resulted in many modified function signatures, so lots of tests had to be edited to pass a connection even if not ultimately used (i.e., a config is being retrieved locally). In these test files, I simply pass in an empty &grpc.AppConn{}. This suffices because, again, the connection does not get used in the code path used by the test. In the few instances that it is used (usually when a mock server is spun up to deliver the config), I use the proper AppConn constructor and defer its closing.

For what's its worth, I've gone through the PR and marked the actual core functionality changes (i.e., not the test changes that are a mere side effect).

Testing RDK successfully reconfigures after config update following various combinations of connecting to/disconnecting from the internet.

bashar-515 · 2025-02-06T17:26:53Z

config/watcher.go

core change

bashar-515 · 2025-02-06T17:27:22Z

robot/impl/local_robot.go

core change

Why is this file a "core change"? Is it because it's part of the "passing down" of the global app conn to config watching? Whereas other files are test changes?

ya sorry should've been more clear, you're right it's just a "non-test" change

bashar-515 · 2025-02-06T17:27:57Z

utils/contextutils/context.go

refactor mentioned earlier

benjirewis

Nice! I've missed the PRs before this one, so may be commenting on pre-existing/already accepted patterns. Ignore my comments in that case.

benjirewis · 2025-02-06T19:11:32Z

components/camera/transformpipeline/detector_test.go

@@ -45,7 +46,8 @@ func writeTempConfig(cfg *config.Config) (string, error) {
 // make a fake robot with a vision service.
 func buildRobotWithFakeCamera(logger logging.Logger) (robot.Robot, error) {
 	// add a fake camera to the config
-	cfg, err := config.Read(context.Background(), artifact.MustPath("components/camera/transformpipeline/vision.json"), logger)
+	cfg, err := config.Read(context.Background(), artifact.MustPath("components/camera/transformpipeline/vision.json"), logger,
+		&grpc.AppConn{})


[nit] You might've used this pattern elsewhere, but I often see nil passed instead of an "empty" struct when we want to communicate to another function that we have no special object for that function to use and it should create its own. Hypothetically, I'd have a small preference for passing nil here and not &grpc.AppConn.

Done! I like this I for some reason glossed over this as an option when trying to figure how to make it apparent that the code path followed by the tests with an empty connection struct didn't actually need/use the conn so I like this

benjirewis · 2025-02-06T19:18:09Z

robot/impl/local_robot.go

Why is this file a "core change"? Is it because it's part of the "passing down" of the global app conn to config watching? Whereas other files are test changes?

benjirewis · 2025-02-06T19:25:53Z

utils/contextutils/context.go

@@ -23,8 +29,20 @@ const (
 	// TimeReceivedMetadataKey is optional metadata in the gRPC response header that correlates
 	// to the time right after the point cloud was captured.
 	TimeReceivedMetadataKey = "viam-time-received"
+
+	readTimeout            = 5 * time.Second


Now that we've moved these three timeout variables to context utils to avoid the import cycle, I think we'll need a) a small sentence describing what "read" they're governing (config reading) and b) to rename the variables to make it more clear what "read" they're governing.

benjirewis · 2025-02-06T19:27:36Z

utils/contextutils/context.go

 )

+// ViamDotDir is the directory for Viam's cached files.
+var ViamDotDir string


The config read timeout vars' presence in this package is acceptable IMO, but I don't love ViamDotDir being here. It's only barely related to context creation; do you have any ideas for a better location @cheukt ?

utils/env.go maybe? but ya, not an easy var to find a location for

Let's do utils/env.go if that SGTY @bashar-515 , I like that better. Thanks, @cheukt .

benjirewis · 2025-02-06T19:28:24Z

web/server/entrypoint.go

@@ -542,7 +542,7 @@ func (s *robotServer) serveWeb(ctx context.Context, cfg *config.Config) (err err
 	}()

 	// watch for and deliver changes to the robot
-	watcher, err := config.NewWatcher(ctx, cfg, s.logger.Sublogger("config"))
+	watcher, err := config.NewWatcher(ctx, cfg, s.logger.Sublogger("config"), s.conn) // TODO(bashar)


TODO(bashar)

Is this not correct as it is?

benjirewis

LGTM! Thanks for the quick edits. Just one more q.

benjirewis · 2025-02-06T23:01:21Z

web/server/entrypoint.go

@@ -173,9 +173,11 @@ func RunServer(ctx context.Context, args []string, _ logging.Logger) (err error)
 		defer pprof.StopCPUProfile()
 	}

+	var appConn rpc.ClientConn


To clarify, appConn remains nil until a connection can be made to app yes? And, you're saying that all code paths you've seen do not try to call *appConn or appConn.foo without verifying that it's non-nil first?

bashar-515 · 2025-02-07T16:40:35Z

LGTM! Thanks for the quick edits. Just one more q.

Correct! appConn is initially nil until we read the local config that contains the info necessary to connect to App (the 'cloud' component of the config). In this case, calls can safely be made to the connection. If there isn't a cloud config, appConn remains nil. This is fine - as far as I've encountered - because all other consumers of the appConn also require a cloud config to exist before using/making calls to appConn. Now, appConn being nil is different from its underlying connection being nil. appConn itself can be non-nil, but it may have a nil underlying connection. This should also be okay bc calls to this underlying connection are protected by a nil check here.

Note this is re: your last question above for some reason github won't let me reply to it directly.

add TODO comment

72f86e2

viambot added the safe to test This pull request is marked safe to test from a trusted zone label Feb 5, 2025

propagate global connection down to config fetch as a function parameter

e177bbb

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 5, 2025

pass connection where necessary

5e95983

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 5, 2025

export context function

c911c20

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 5, 2025

checkpoint

8bc2348

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 5, 2025

update reader_test.go file

e93dfce

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 6, 2025

update config_test.go file

06d6a32

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 6, 2025

update entrypoint_test.go file

8c867bf

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 6, 2025

update all builtin_test.go files

65920bb

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 6, 2025

update tests

868e134

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 6, 2025

prevent nil pointer dereference

822ed05

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 6, 2025

lint

d030c30

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 6, 2025

bashar-515 commented Feb 6, 2025

View reviewed changes

config/watcher.go Outdated

Copy link

Member Author

bashar-515 Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

core change

bashar-515 commented Feb 6, 2025

View reviewed changes

utils/contextutils/context.go

Copy link

Member Author

bashar-515 Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refactor mentioned earlier

bashar-515 added 3 commits February 6, 2025 13:29

add TODO comment

4eb2805

use global connection to app in cert fetch

0d73435

lint

f2d84f5

benjirewis reviewed Feb 6, 2025

View reviewed changes

bashar-515 added 5 commits February 6, 2025 14:55

use nil in place of empty connection

877c5eb

remove TODO comment

80556bd

move ViamDotDir to utils

892d072

make timeouts more clear

4b7c683

remove TODO comment

92f33c5

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 6, 2025

prefer passing nil instead of empty connection struct

2264356

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 6, 2025

pass nil again

0caf8d7

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Feb 6, 2025

bashar-515 requested a review from benjirewis February 6, 2025 21:34

benjirewis approved these changes Feb 6, 2025

View reviewed changes

bashar-515 merged commit edca01a into viamrobotics:main Feb 7, 2025
16 checks passed

bashar-515 deleted the RSDK-8291 branch February 7, 2025 16:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RSDK-8291] Use Global Connection to App in Config Watcher #4773

[RSDK-8291] Use Global Connection to App in Config Watcher #4773

bashar-515 commented Feb 5, 2025 •

edited

Loading

bashar-515 Feb 6, 2025

bashar-515 Feb 6, 2025

benjirewis Feb 6, 2025

bashar-515 Feb 6, 2025

bashar-515 Feb 6, 2025

benjirewis left a comment •

edited

Loading

benjirewis Feb 6, 2025

bashar-515 Feb 6, 2025

benjirewis Feb 6, 2025

benjirewis Feb 6, 2025

benjirewis Feb 6, 2025

cheukt Feb 6, 2025

benjirewis Feb 6, 2025

benjirewis Feb 6, 2025

benjirewis left a comment

benjirewis Feb 6, 2025

bashar-515 commented Feb 7, 2025

[RSDK-8291] Use Global Connection to App in Config Watcher #4773

[RSDK-8291] Use Global Connection to App in Config Watcher #4773

Conversation

bashar-515 commented Feb 5, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjirewis left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjirewis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bashar-515 commented Feb 7, 2025

bashar-515 commented Feb 5, 2025 •

edited

Loading

benjirewis left a comment •

edited

Loading