Feature request

[Swoorup]

Does this support inserting historical order book data?

[bolobox]

+1 for order book storage support in marketstore!

[umitanuki]

Yes that's what we want, too! The data format is going be more complicated so we need some design time.

[athenawisdoms]

This is an excellent idea!

Any updates on order book data integration, please?

If not, any recommendations on how to currently best store historical order book data?

@umitanuki

[goodboy]

@athenawisdoms you might want to give techtonicDB a try for an out of the box solution with an excellent storage format.

However, I am already story tick data using the <symbol>/1Sec/TICK time bucket (note the Nanoseconds subtleties from #325) and it is working quite well. Adding an additional Level or Price field for each row entry should give you something similar to what techtonic provides though of course with a much less compact format.

[athenawisdoms]

@goodboy Great recommendation, will be sure to try it out.

If we want to store both trades and order book data, will you suggest using marketstore for trades and techtonicDB for order book? Or sticking to just one (i.e. marketstore) like you have done?

Additionally. have you looked into the data compression ratio for order book data when using marketstore the way you are currently storing, compared to a more widely used time series (eg. timescaledb, influxdb)?

Thanks!

[goodboy]

@athenawisdoms I actually am using both at the moment (techtonicDB as well just don't have code up yet). I'm currently delegating to underlying broker systems to store trade history so that hasn't been a requirement yet.

have you looked into the data compression ratio for order book data when using marketstore the way you are currently storing, compared to a more widely used time series (eg. timescaledb, influxdb)?

I haven't in detail but the marketstore design docs might provide insight. I have done some reading on both timescaledb and influxdb. You can see a small list of options we've been auditing here: pikers/piker#90

[goodboy]

Pretty sure this pertains to the new tick to bar conversion that's just been added in #399.

Tick is usually the level 1 of the book (aka the bid/ask spread) so if tick aggregation has now been added then we have a very basic top of book time series going on.