draft_protocol multifetch

RFC multifetch protocol

This text follows RFC-like language. Refer to http://rfc.net/rfc2119.html.

'''Design goals:'''

• Simple ability for plugins to send sub-5min sampled values
• Storing protocol for [wiki:spec-spooldir spoolfetch repository]
• Simplicity & transparency: storing by append-only

Existing model

Actually each plugin gives only one value.

Advantages

• Lightweight plugin
• Ability to have a streaming plugin

Disadvantages

• For very high sampling rates (<< 1m) the overhead is noticable.

Proposed new model

A plugin MAY just send many timed-values at once. It MUST then prefix the values with an epoch. This format is mostly used for an efficient spoolfetch storage : the format is plain text and append-only. The different values MAY be separated with a blank line for easier parsing.

The epoch MUST be only increasing.

Streaming plugins

This enables having a streaming plugin : it never ends when asked for values.

It would be very useful for the cpu plugin that would open a streaming vmstat, massage its output, and restream the output. The advantage would be to have a very little overhead and then avoiding regular monitoring spikes.

Protocol example

field1.value 1234567890:1
field2.value 1234567890:2
field3.value 1234567890:3

field1.value 1234567891:10
field2.value 1234567891:20
field3.value 1234567891:30

field1.value 1234567900:10
field2.value 1234567900:60
field3.value 1234567900:7.3423