-
Notifications
You must be signed in to change notification settings - Fork 745
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial version of defining the interfaces to accept metrics #15913
Changes from 2 commits
2c63137
3dd931b
7d24843
f8fb42f
b48641a
2d06ebd
9650f9d
9e0a294
602f06e
a4cf9fd
6fcf355
31f56d5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,147 @@ | ||
# This file defines the interfaces that snappi tests accept external metrics. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the definitions of the metric names and meta are missing in the file, we need to get them defined and show a unified format. this will be used for crafting the dashboards. |
||
|
||
# Metrics data are organized into the hierarchies below | ||
# TestMetrics | ||
# ├── TestID | ||
# └── DeviceMetrics | ||
# ├── DeviceID | ||
# └── Metric | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. create a generic Metric class that represents a single metric, which contains:
class Metric...:
def __init__(name, ...., reporter):
reporter.add_metric(self)
....
class GaugeMetric(Metric):
def __init__(name, ...., reporter):
super.__init__(...)
self.value = 0
def set(v):
self.value = v
....
reporter = MetricReporterFactory(...).build()
port_rx = GaugeMetric(...., reporter)
port_rx.set(123)
reporter.report(time) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hence, ultimately the final code for people to use would be: metrics = {
"PortRx" = GaugeMetric(......, reporter)
....
}
for r in csv:
for c in r:
metric[c.title].set(c.value)
reporter.report(time) |
||
# ├── Name | ||
# ├── Description | ||
# ├── Unit | ||
# ├── metadata | ||
# └── data | ||
# └── Gauge | ||
# | ||
# A TestMetrics has its ID and a list of DeviceMetrics objects. | ||
# A DeviceMetrics has its ID and a list of Metric objects. | ||
# A Metric has several attributes and data. So far we only have Gauge type data. | ||
# A Gauge has a list of NumberDataPoint objects. | ||
# A NumberDataPoint has its label, value, flags and the timestamp at which the data was collected. | ||
# | ||
# | ||
# +-----------+ | ||
# |DataPoint 1| | ||
# | +-----+ | | ||
# | |label| | | ||
# +-----+ | +-----+ | | ||
# | 1 |---> | +-----+ | | ||
# +-----+ | |value| | | ||
# | . | | +-----+ | | ||
# | . | | +-----+ | | ||
# | . | | |flags| | | ||
# | . | | +-----+ | | ||
# | . | +-----------+ | ||
# | . | . | ||
# | . | . | ||
# | . | . | ||
# | . | +-----------+ | ||
# | . | |DataPoint M| | ||
# | . | | +-----+ | | ||
# | . | | |label| | | ||
# +-----+ | +-----+ | | ||
# | M |---> | +-----+ | | ||
# +-----+ | |value| | | ||
# | +-----+ | | ||
# | +-----+ | | ||
# | |flags| | | ||
# | +-----+ | | ||
# +-----------+ | ||
|
||
|
||
|
||
|
||
from typing import List, Dict, Union | ||
sm-xu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
############################## Accept Metrics ############################## | ||
|
||
# All metrics of one TestMetrics object are from the same testbed runing the same | ||
# software version. They are also from the same test case identified by test_run_id. | ||
class TestMetrics: | ||
def __init__(self, testbed_name, os_version, testcase_name, test_run_id): | ||
self.testbed_name = testbed_name | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. all these fields can be moved to reporter, since it is shared by everyone. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. TestMetrics itself can be removed, once we add the per metric class. |
||
self.os_version = os_version | ||
self.testcase_name = testcase_name | ||
self.test_run_id = test_run_id | ||
self.device_metrics = [] | ||
|
||
def add_device_metrics(self, device_metric): | ||
self.device_metrics.append(device_metric) | ||
|
||
def __repr__(self): | ||
return f"TestMetrics(test={self.test}, device_metrics={self.device_metrics})" | ||
|
||
|
||
# All metrics of one DeviceMetrics object are from the same device identified by device_id. | ||
class DeviceMetrics: | ||
def __init__(self, device_id): | ||
self.device_id = device_id | ||
self.metrics = [] | ||
|
||
def add_metric(self, metric): | ||
self.metrics.append(metric) | ||
|
||
def __repr__(self): | ||
return f"DeviceMetrics(device={self.device}, metrics={self.metrics})" | ||
|
||
|
||
# All metrics of one Metric object belong to the same category tagged by metric name, | ||
# e.g., psu info, temperature info, port counters | ||
class Metric: | ||
def __init__(self, name, description, unit, data_points, metadata = None): | ||
self.name = name # Metric name (e.g., psu, temperature) | ||
self.description = description # Metric description | ||
self.unit = unit # Metric unit (e.g., seconds, bytes) | ||
self.data = data # Can be Gauge only | ||
self.metadata = metadata or {} # e.g. port_id, psu_id, default to an empty dictionary if None | ||
|
||
def __repr__(self): | ||
return (f"Metric(name={self.name}, description={self.description}, " | ||
f"unit={self.unit}, data={self.data})") | ||
|
||
|
||
class Gauge: | ||
def __init__(self, time_unix_nano: int): | ||
self.time_unix_nano = time_unix_nano # UNIX Epoch time in nanoseconds | ||
self.data_points = [] # List of NumberDataPoint objects | ||
|
||
def add_data_point(self, data_point): | ||
self.data_points.append(data_point) | ||
|
||
def __repr__(self): | ||
return f"Gauge(data_points={self.data_points})" | ||
|
||
|
||
class NumberDataPoint: | ||
def __init__(self, label: List[Dict[str, str]], value: Union[int, float], flags: int = None): | ||
self.label = label # The key of key-value pairs in dictionaries | ||
self.value = value # Metric value (can be double or integer) | ||
self.flags = flags # Optional flags | ||
|
||
def __repr__(self): | ||
return (f"NumberDataPoint(label={self.label}, " | ||
f"time_unix_nano={self.time_unix_nano}, value={self.value}, flags={self.flags})") | ||
|
||
############################## Report Metrics ############################## | ||
|
||
class MetricReporterFactory: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. move factory to another file, so we can override easily. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. with this change, we can do this in another file: class MetricReporterFactory:
def create_metrics_reporter(self):
return OtelMetricReporter(...)
class OtelMetricReporter:
def emit(....):
# Real implementation goes here, which each customer can define their own. |
||
def __init__(self, testbed_name, testcase_name, test_run_id): | ||
self.testbed_name = testbed_name | ||
self.testcase_name = testcase_name | ||
self.test_run_id = test_run_id | ||
|
||
def create_metrics_reporter(self): | ||
# Create MetricsReporter here. | ||
pass | ||
|
||
|
||
class MetricsReporter: | ||
def __init__(self, testbed_name, testcase_name, test_run_id): | ||
self.testbed_name = testbed_name | ||
self.testcase_name = testcase_name | ||
self.test_run_id = test_run_id | ||
|
||
def emit_metrics(metrics: TestMetrics): | ||
# to be implemented | ||
pass |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All common label names are missing too, e.g.: PortId, QueueId, PSUId....
otherwise it will be very hard to create unified dashboard, because each tests could use its own names, and causing problems in filters.