L2 subsystem(s)

[goodboy]

Depth of book, otherwise known as L2 data. is something that's been on the list todo for quite a while.
We now not only have the data processing and streaming infra for it but also the UI components to display it efficiently in real-time on charts.

I originally had though about collecting and storing L2 using marketstore tick records as per #93 and #142. When you look at our tick schema it makes you think, ok so we need to store info at each level and I guess this goes all in one time series array?, well basically except there's already a project with a superior technique.

The forward gist is that techtonicdb is a much better fit with it's super compact DFT format:
(timestamp, seq, is_trade, is_bid, price, size).

Component design, subsystem todos

• we need an ability to spawn a local instance using the docker container
• a client can talk to techtonic over TCP as per the messaging spec
  • we probably want a trio-fied client to do this (which should be trivial using trio.SocketStream).
  • implement a client in our repo first, get it working with everything else, and then we can break it out later (todo make issue for that if we want)
• we'll want to extend the existing L1 feed UI to show bars and/or lines at each book-price level (this could also get switched later to a curve like in cryptowatch if peeps prefer that)

Further resources

• the set of python scripts in the techtonicdb repo
• example L2 plotting code in python giving an idea of how to display levels using pandas and matplotlib
  • pretty sure it originally came from this blog post (Gradient Trader Part 2) from the author
    • notice the note at the end that will possibly be worth follow up:

    In the future I plan on porting such visualization to react-stockcharts so I can interactively explore data. The algorithms are already here, what is left to be done is changing matplotlib calls to d3. If you would like to collaborate with me on this, please email me.

• an example trading bot
• a small monitoring script also in python
• a short blog post by the author on microstructure backtesting which gives some intuition for what real-time charting might look like
• the original post by the author that was pre-cursor to the creation of the techtonicdb rust stack
• an ffi.py script for reading raw data from the db's storage file