-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature]: Support pickle
and copy.deepcopy
for Sentences
#3243
Labels
feature
A new feature
Comments
helpmefindaname
added a commit
that referenced
this issue
May 22, 2023
helpmefindaname
added a commit
that referenced
this issue
Jul 17, 2023
helpmefindaname
added a commit
that referenced
this issue
Aug 7, 2023
helpmefindaname
added a commit
that referenced
this issue
Aug 8, 2023
helpmefindaname
added a commit
that referenced
this issue
Oct 23, 2023
helpmefindaname
added a commit
that referenced
this issue
Mar 29, 2024
helpmefindaname
added a commit
that referenced
this issue
Aug 30, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Problem statement
Serializing and deserializing sentences with
pickle
is a common practice when using Flair in multiprocessing systems, e.g. with the Ray library. Similarly, creating deep copies of objects is a usually natural feature. Unfortunately, loading a pickled sentence withpickle
or creating a deep copy withcopy.deepcopy
fails for sentences with span or relation annotations.Example:
Related to #3191.
Solution
It would be great if sentences with arbitrary annotations were pickable or at least serializable in any manner and copyable.
Additional Context
The reason for the
TypeError
is sentences annotated with spans or relations contain references to these, where their implementations use the__new__
method.flair/flair/data.py
Line 550 in 5c53e35
flair/flair/data.py
Line 609 in 5c53e35
To fix this the
pickle
(andcopy
) module require implementing a__getnewargs__
or__getnewargs_ex__
method that provides the arguments for__new__
when unpickling an object.The text was updated successfully, but these errors were encountered: