You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Indexing pipelines currently only support textual data. There are more kinds of data, images, byte slices, various forms of structured data, binary, etcetera.
Describe the solution you'd like
Proposal is to make nodes generic over the inner value, i.e. a Node<Text>, Node, Node, Node` (or their underlying primitives). Added benefit that transformers can restrict the kind of data they can work with.
Additionally, for an initial release:
Pipeline must be able to change the node type after any step to a new type
All current steps have to be updated, extra care for existing steps that they work with all / the ones they support
Debug has to be implemented correctly, nobody wants binary data in their logs by accident
Describe alternatives you've considered
One alternative is an enum. However, that means handling the enum in every transformation step and limits the typesafety.
Additional context
See #356 for a cool idea that could use this
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Indexing pipelines currently only support textual data. There are more kinds of data, images, byte slices, various forms of structured data, binary, etcetera.
Describe the solution you'd like
Proposal is to make nodes generic over the inner value, i.e. a
Node<Text>,
Node,
Node,
Node` (or their underlying primitives). Added benefit that transformers can restrict the kind of data they can work with.Additionally, for an initial release:
Describe alternatives you've considered
One alternative is an enum. However, that means handling the enum in every transformation step and limits the typesafety.
Additional context
See #356 for a cool idea that could use this
The text was updated successfully, but these errors were encountered: