Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

making either store or config #30

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 31 additions & 4 deletions pydantic_redis/_shared/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from pydantic.fields import ModelPrivateAttr
from redis import Redis
from redis.asyncio import Redis as AioRedis
from pydantic import ConfigDict, BaseModel
from pydantic import ConfigDict, BaseModel, model_validator
from redis.commands.core import Script, AsyncScript

from ..config import RedisConfig
Expand All @@ -32,7 +32,7 @@ class AbstractStore(BaseModel):
"""

name: str
redis_config: RedisConfig
redis_config: Optional[RedisConfig] = None
redis_store: Optional[Union[Redis, AioRedis]] = None
life_span_in_seconds: Optional[int] = None
select_all_fields_for_all_ids_script: Optional[Union[AsyncScript, Script]] = None
Expand All @@ -51,11 +51,29 @@ class AbstractStore(BaseModel):
def __init__(
self,
name: str,
redis_config: RedisConfig,
redis_config: Optional[RedisConfig] = None,
redis_store: Optional[Union[Redis, AioRedis]] = None,
life_span_in_seconds: Optional[int] = None,
**data: Any,
):
"""
A store provides all functionality required to quickly persist and retrieve
pydantic objects into and from a Redis store.

This store will accept the configuration (in the form of a `RedisConfig`, and
create a new Redis instance according to that configuration. Alternatively,
one can pass an existing Redis instance to be used. One can only pass one ot
these.

:param name: The name of the store.
:param redis_config: A Redis configuration object. This will be used to create a
new Redis instance, using the parameters configured.
:param redis_store: An existing Redis instance to use. Can only be passed when no
`redis_config` is passed.
:param life_span_in_seconds: The default lifespan in seconds that the store will use
:param data: an other keyword argument that will be passed to the constructor of
the object
"""
super().__init__(
name=name,
redis_config=redis_config,
Expand All @@ -64,9 +82,18 @@ def __init__(
**data,
)

self.redis_store = self._connect_to_redis()
# because of validator we can safely assume either `redis_store` is passed or `redis_config`
if self.redis_store is None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to reintroduce the ambiguity we removed when we removed the redis_store argument.

For instance someone might set both redis_config and redis_store with different redis databases.
It is not quickly apparent to anyone that the redis_store has higher precedence over the redis_config. This might cause that person a lot of frustration when they don't see the data where they expect.

We need to have a single way of passing the redis instance so that there is never a possibility of passing two redis instances.

Options:

  1. Extend RedisConfig to allow passing a pre-constructed redis instance.
    Con: It might require a lot of complicated code to ensure two instances are never passed to it (the RedisConfig).
    Pro: The signature of AbstractStore is untouched; no breaking change.
  2. Change redis_config argument to redis_store: Union[Redis, AioRedis, RedisConfig].
    Con: The signature of AbstractStore is changed; this is a breaking change.
    Pro: The code is simple and ensures that only one redis instance is ever passed to the store.
    Pro: The name redis_store as opposed to redis_config makes intuitive sense for a parameter that can either be a redis instance or a RedisConfig
  3. Change redis_config argument to redis_config: Union[Redis, AioRedis, RedisConfig].
    Pro: The signature of AbstractStore is unchanged; no breaking change.
    Pro: The code is simple and ensures that only one redis instance is ever passed to the store.
    Con: The name redis_config as opposed to redis_store does not make it apparent it can either be a redis instance or a RedisConfig especially since there is an actual class called "RedisConfig"

...There could be other options but personally, I am leaning more on option 3 then adding some extra documentation specifying that redis_config can also receive a redis instance. This way, the many applications currently running out there that were built basing on our original documentation do not fail suddenly on upgrading.

I also expect that not many people except the "power users" (as you commented earlier) will want to have a way of passing in their pre-constructed redis instance. And since they are "power users", they definitely can comb through the documentation for such finer control options :-)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest to stick with the names you have: redis_config and redis_store. The validator does this:

    @model_validator(mode='after')
    def _config_or_redis(self):
        if (self.redis_config is None) and (self.redis_store is None):
            raise ValueError("Must provide one of redis_config or redis_store")
        if (self.redis_config is not None) and (self.redis_store is not None):
            raise ValueError("Cannot provide both redis_config and redis_store")

So if a programmer happens to pass in values for both the redis_config and the redis_store, they will be given a ValueError, explaining they can only pass in one of them.

This way the parameter names remain descriptive.

I will add a docstring that explains this better.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just added the following DocString:

    def __init__(
        self,
        name: str,
        redis_config: Optional[RedisConfig] = None,
        redis_store: Optional[Union[Redis, AioRedis]] = None,
        life_span_in_seconds: Optional[int] = None,
        **data: Any,
    ):
        """
        A store provides all functionality required to quickly persist and retrieve
        pydantic objects into and from a Redis store.

        This store will accept the configuration (in the form of a `RedisConfig`, and
        create a new Redis instance according to that configuration. Alternatively,
        one can pass an existing Redis instance to be used. One can only pass one ot
        these.

        :param name: The name of the store.
        :param redis_config: A Redis configuration object. This will be used to create a
        new Redis instance, using the parameters configured.
        :param redis_store: An existing Redis instance to use. Can only be passed when no
        `redis_config` is passed.
        :param life_span_in_seconds: The default lifespan in seconds that the store will use
        :param data: an other keyword argument that will be passed to the constructor of
        the object
        """

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This validator is demonstrated to work in the test test_cannot_provide_both as follows:

def test_cannot_provide_both(unused_tcp_port: int):
    redis = redislite.Redis(host="localhost", port=unused_tcp_port)
    config = RedisConfig(host="localhost", port=unused_tcp_port)

    with pytest.raises(ValueError):
        _ = Store(
            name="redis-or-config",
            redis_config=config,
            redis_store=redis,
        )

Copy link
Contributor

@Tinitto Tinitto Jul 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need the library to be straight forward. Adding two ways of initializing the Store needs to be last resort.
For now, just extend the Store

class MyStore(AbstractStore): # or MyStore(syncio.Store): or or MyStore(asyncio.Store)
    """My custom store"""

    def __init__(
        self, name: str, redis_config: RedisConfig, redis_store: Redis, **data: Any
    ):
        self._redis_store = redis_store
        super().__init__(name, redis_config, **data)

    def _connect_to_redis(self) -> redis.Redis:
        """Connects the store to redis.

        See base class.
        """
        return self._redis_store


store = MyStore(
    name="foobar", redis_config=RedisConfig(), redis_store=your_redis_instance
)

self.redis_store = self._connect_to_redis()
self._register_lua_scripts()

@model_validator(mode='after')
def _config_or_redis(self):
if (self.redis_config is None) and (self.redis_store is None):
raise ValueError("Must provide one of redis_config or redis_store")
if (self.redis_config is not None) and (self.redis_store is not None):
raise ValueError("Cannot provide both redis_config and redis_store")

def _connect_to_redis(self) -> Union[Redis, AioRedis]:
"""Connects the store to redis.

Expand Down
44 changes: 44 additions & 0 deletions test/test_redis_or_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import pytest
import redislite
from redis.client import Redis

from pydantic_redis import Store, RedisConfig


def test_need_config_or_redis():
with pytest.raises(ValueError):
_ = Store(name="no-redis-or-config")


def test_cannot_provide_both(unused_tcp_port: int):
redis = redislite.Redis(host="localhost", port=unused_tcp_port)
config = RedisConfig(host="localhost", port=unused_tcp_port)

with pytest.raises(ValueError):
_ = Store(
name="redis-or-config",
redis_config=config,
redis_store=redis,
)

def test_config_creates_new():
store = Store(
name="redis-or-config",
redis_config=RedisConfig(
host="localhost",
port=6379,
db=0,
),
)
assert store.redis_store is not None
assert isinstance(store.redis_store, Redis)


def test_store_creates_new(unused_tcp_port: int):
redis = redislite.Redis(host="localhost", port=unused_tcp_port)
pydantic_redis_store = Store(
name="redis-or-config",
redis_store=redis,
)

assert pydantic_redis_store.redis_store == redis