Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pydantic root_validator usage #1105

Closed
wants to merge 2 commits into from

Conversation

thinkall
Copy link
Collaborator

@thinkall thinkall commented Jul 1, 2023

Why are these changes needed?

pydantic new version is released today, thus current code will raise below error:

https://github.com/microsoft/FLAML/actions/runs/5428592991/jobs/9872924665?pr=1097#step:12:98

ERROR test/autogen/test_math_user_proxy_agent.py - pydantic.errors.PydanticUserError: If you use `@root_validator` with pre=False (the default) you MUST specify `skip_on_failure=True`. Note that `@root_validator` is deprecated and should be replaced with `@model_validator`.

This PR added skip_on_failure=True to fix the error.

Related issue number

Checks

@thinkall
Copy link
Collaborator Author

thinkall commented Jul 1, 2023

ray has an issue with latest pydantic as well:
https://github.com/microsoft/FLAML/actions/runs/5428749030/jobs/9873215390?pr=1105#step:12:3174

_______________________________ test_consistency _______________________________

self = <ray.tune.execution.trial_runner.TrialRunner object at 0x7f02a18d9970>
next_trial = evaluate_config_a4cb7aa6

    def _wait_and_handle_event(self, next_trial: Optional[Trial]):
        try:
            # Single wait of entire tune loop.
>           event = self.trial_executor.get_next_executor_event(
                self._live_trials, next_trial is not None
            )

/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/tune/execution/trial_runner.py:1369: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/tune/execution/ray_trial_executor.py:1119: in get_next_executor_event
    self._stage_and_update_status(live_trials)
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/tune/execution/ray_trial_executor.py:318: in _stage_and_update_status
    self._resource_manager.request_resources(resource_request=resource_request)
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/air/execution/resources/placement_group.py:143: in request_resources
    future = pg.ready()
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/util/placement_group.py:78: in ready
    return bundle_reservation_check.options(
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/remote_function.py:225: in remote
    return func_cls._remote(args=args, kwargs=kwargs, **updated_options)
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/util/tracing/tracing_helper.py:301: in _invocation_remote_span
    return method(self, args, kwargs, *_args, **_kwargs)
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/remote_function.py:405: in _remote
    return invocation(args, kwargs)
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/remote_function.py:380: in invocation
    object_refs = worker.core_worker.submit_task(
python/ray/_raylet.pyx:2008: in ray._raylet.CoreWorker.submit_task
    ???
python/ray/_raylet.pyx:2012: in ray._raylet.CoreWorker.submit_task
    ???
python/ray/_raylet.pyx:427: in ray._raylet.prepare_args_and_increment_put_refs
    ???
python/ray/_raylet.pyx:418: in ray._raylet.prepare_args_and_increment_put_refs
    ???
python/ray/_raylet.pyx:464: in ray._raylet.prepare_args_internal
    ???
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/_private/worker.py:601: in get_serialization_context
    context_map[job_id] = serialization.SerializationContext(self)
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/_private/serialization.py:151: in __init__
    serialization_addons.apply(self)
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/ray/util/serialization_addons.py:58: in apply
    register_pydantic_serializer(serialization_context)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

serialization_context = <ray._private.serialization.SerializationContext object at 0x7f02a18da430>

    @DeveloperAPI
    def register_pydantic_serializer(serialization_context):
        try:
            import pydantic.fields
        except ImportError:
            return
    
        # Pydantic's Cython validators are not serializable.
        # https://github.com/cloudpipe/cloudpickle/issues/408
        serialization_context._register_cloudpickle_serializer(
>           pydantic.fields.ModelField,
            custom_serializer=lambda o: {
                "name": o.name,
                # outer_type_ is the original type for ModelFields,
                # while type_ can be updated later with the nested type
                # like int for List[int].
                "type_": o.outer_type_,
                "class_validators": o.class_validators,
                "model_config": o.model_config,
                "default": o.default,
                "default_factory": o.default_factory,
                "required": o.required,
                "alias": o.alias,
                "field_info": o.field_info,
            },
            custom_deserializer=lambda kwargs: pydantic.fields.ModelField(**kwargs),
        )
E       AttributeError: module 'pydantic.fields' has no attribute 'ModelField'

@yiranwu0
Copy link
Contributor

yiranwu0 commented Jul 1, 2023

Yes, pydantic just updated to the new version of v2, I think we can stay with v1 for now

@thinkall
Copy link
Collaborator Author

thinkall commented Jul 1, 2023

Yes, pydantic just updated to the new version of v2, I think we can stay with v1 for now

I see you already fixed the pydantic version in another PR. So, I'll close this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants