HLD for Redis Client Manager #1810
Conversation
|
|
||
| #### RedisClient(db DBNum) | ||
|
|
||
| This RedisClient function takes in a database number, defined in [db.go](https://github.com/sonic-net/sonic-mgmt-common/blob/master/translib/db/db.go#L136), and returns an existing Redis Client to the caller. This Redis Client can be shared across multiple goroutines and has a connection pool. It cannot be used for [Transactional Redis Operations](https://redis.io/docs/latest/develop/interact/transactions/) (such as MULTI, EXEC, PSUBSCRIBE, etc...). |
There was a problem hiding this comment.
Today redis clients are created by db.NewDB(), which is called per translib API (Get, Create, Delete etc). Each translib API handling will involve multiple redis operations of different kinds. All of them are using the same redis client created during db.NewDB() call today. How do you propose to predict the kind of redis operations that will be performed by the app code during db.NewDB() call itself -- which will done by the translib infra?
Are you planning to use IsWriteDisabled flag? If yes, please mention so instead of listing redis operations.
There was a problem hiding this comment.
I introduced a new DB option called TransactionsRequired (boolean). Callers of NewDB will need to set this flag in order to get a unique Redis client in the DB that is returned. This will be handled during the refactoring.
Using IsWriteDisabled is another option that we can consider. However, I created this new option to be more explicit.
Adding this detail to the doc.
|
|
||
| #### Refactoring Callsites | ||
|
|
||
| In all places where a Redis Client is not used to perform a transactional operation, the call will be refactored to use RedisClient(). This will result in the caller now using a shared Redis Client with a connection pool. This will significantly reduce the number of new Redis Clients and connections to Redis. |
There was a problem hiding this comment.
Please list the callsites you are planning to change.. Today the DB object and its redis client are created once per translib API call and not by the apps. How the "kind of redis operations" performed by apps are predicted by the infra?
There was a problem hiding this comment.
Updated the HLD to include callsites. In general, we assume that GET and SUBSCRIBE related Translib APIs do not need a transactional Redis Client and SET related APIs do. OnChange Subscriptions also use a transactional Redis Client to subscribe to keyspace notifications.
Outside of these Translib APIs, transactional Redis Clients are used.
|
|
||
| ### Restrictions/Limitations | ||
|
|
||
| RCM lives in the translib/db package. This package imports CVL, so refactoring calls in CVL to use RCM would create a circular dependency. |
There was a problem hiding this comment.
Can be resolved by placing the RCM code in a separate package parallel to translib & cvl
There was a problem hiding this comment.
Thanks for the suggestion. I think it makes the most sense for this code to live in the sonic-mgmt-common/translib/db directory because the API will be integrated with the db_redis_opts.go and db_stats.go. We can consider moving it out of this directory if necessary.
This HLD describes a design for using Redis connection pools in sonic-mgmt-common to reduce stress on the Redis Unix/TCP socket.