introduction

Metrictank is a cassandra-backed, metrics2.0 based, multi-tenant timeseries database for Graphite and friends

metrictank is a timeseries database, inspired by the Facebook gorilla paper. Most notably, it employs the compression mechanism described in the paper to dramatically lower storage overhead, as well as data chunking to lower the load on cassandra. Graphite users are first-class citizens.

status

While raintank has been running metrictank in production since december 2015, there are still plenty of kinks to work out and bugs to fix. It should be considered an alpha project.

limitations

• no performance/availability isolation between tenants per instance. (only data isolation)
• clustering is basic: statically defined peers, master promotions are manual, etc. See clustering for more.
• minimum computation locality: we move the data from storage to processing code, which is both metrictank and graphite.
• the datastructures can use performance engineering. A Go GC issue may occassionally inflate response times.
• the native input protocol is inefficient. Should not send all metadata with each point.
• we use metrics2.0 in native input protocol and indexes, but barely do anything with it yet.
• for any series you can't write points that are earlier than previously written points. (unless you restart MT)

interesting design characteristics (feature or limitation.. up to you)

• upgrades / process restarts requires running multiple instances (potentially only for the duration of the maintenance) and re-assigning the primary role. Otherwise data loss of current chunks will be incurred. See operations guide
• only float64 values. Ints and bools currently stored as floats (works quite well due to the gorilla compression), No text support.
• only uint32 unix timestamps in second resolution. For higher resolution, consider streaming directly to grafana
• no data locality: doesn't seem needed yet to put related series together.

main features

• 100% open source
• graphite is a first class citizen. As of graphite-1.0.1, metrictank can be used as a graphite CLUSTER_SERVER.
• accurate, flexible rollups by storing min/max/sum/count (which also gives us average). So we can do consolidation (combined runtime+archived) accurately and correctly, unlike most other graphite backends like whisper
• metrictank acts as a writeback RAM cache for recent data.
• flexible tenancy: can be used as single tenant or multi tenant. Selected data can be shared across all tenants.
• input options: carbon, metrics2.0, kafka. (soon: json or msgpack over http)
• guards against excessive data requests
• efficient data compression and efficient use of Cassandra.

Docs

installation, configuration and operation.

• Overview
• Quick start using docker
• Installation guides
• Configuration
• Memory server
• Compression tips
• Cassandra
• Kafka
• Inputs
• Metrics
• Operations
• Tools

features in-depth

• Clustering
• Consolidation
• Multi-tenancy
• HTTP api
• Graphite
• Metadata
• Tags
• Usage reporting

Other

• Development
• Community
• Roadmap
• Faq

Releases and versioning

• releases and changelog

• we aim to keep master stable and vet code before merging to master

• We're pre-1.0 but adopt semver for our 0.MAJOR.MINOR format. The rules are simple:

  • MAJOR version for incompatible API or functionality changes
  • MINOR version when you add functionality in a backwards-compatible manner, and

  We don't do patch level releases since minor releases are frequent enough.

License

Copyright 2016 Dieter Plaetinck, Anthony Woods, Jeremy Bingham, Damian Gryski, raintank inc

This software is distributed under the terms of the GNU Affero General Public License.