You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, serialization output is directed to a Write in both the V2 and V2 + DEFLATE serializers. This is a very flexible abstraction, but the serialization formats dictate a length prefix, which means that output must be buffered before writing to the writer so that we can drop in the final length in the appropriate spot.
This buffering is especially sad in the DEFLATE case, since we already have a Vec<u8> that will hold the uncompressed serialization, but we can only expose it to the underlying V2 serializer as a Write, so the uncompressed bytes get written to V2's buffer, then copied into V2 + DEFLATE's buffer, then compressed into another buffer, then written to the user-provided writer.
Some options:
Functions for Write or Vec<u8>, where the former buffers into an internal Vec<u8>
Functions for Write or Write + Seek. Need to benchmark this to see if it's measurably different from the Vec<u8> case. If it isn't, this seems preferable as it is more general.
Give up on Write and only serialize into either Write + Seek or a Vec<u8>. Maybe supporting I/O streams that don't seek is a sufficiently niche case that it's not worth creating an easy path for? On one hand, I really want to encourage people to use these formats as a wire protocol in a monitoring system. On the other hand, maybe such a protocol would use a container format like protobuf around the serialized histogram anyway, and it's a waste of API complexity budget to support simple Write usage.
The text was updated successfully, but these errors were encountered:
Currently, serialization output is directed to a
Write
in both the V2 and V2 + DEFLATE serializers. This is a very flexible abstraction, but the serialization formats dictate a length prefix, which means that output must be buffered before writing to the writer so that we can drop in the final length in the appropriate spot.This buffering is especially sad in the DEFLATE case, since we already have a
Vec<u8>
that will hold the uncompressed serialization, but we can only expose it to the underlying V2 serializer as aWrite
, so the uncompressed bytes get written to V2's buffer, then copied into V2 + DEFLATE's buffer, then compressed into another buffer, then written to the user-provided writer.Some options:
Write
orVec<u8>
, where the former buffers into an internalVec<u8>
Write
orWrite + Seek
. Need to benchmark this to see if it's measurably different from theVec<u8>
case. If it isn't, this seems preferable as it is more general.Write
and only serialize into eitherWrite + Seek
or aVec<u8>
. Maybe supporting I/O streams that don't seek is a sufficiently niche case that it's not worth creating an easy path for? On one hand, I really want to encourage people to use these formats as a wire protocol in a monitoring system. On the other hand, maybe such a protocol would use a container format like protobuf around the serialized histogram anyway, and it's a waste of API complexity budget to support simpleWrite
usage.The text was updated successfully, but these errors were encountered: