This repository has been archived by the owner on Nov 1, 2024. It is now read-only.

Modularize aggregation metric code to enable easy adding of more aggregation metrics #761

Open

TristanThrush opened this issue Oct 6, 2021 · 1 comment

Assignees

Milestone

Contributor

TristanThrush commented Oct 6, 2021

The only possible aggregation metric is dynascore. After modularizing this code, it would be good to add z-score, plain old averaging, and perf-only.

TristanThrush added this to the Bigger tickets milestone

kokrui self-assigned this

kokrui mentioned this issue

Make dynascore metric weights settable in task owner interface #763

Closed

Contributor

kokrui commented Oct 15, 2021

just to confirm:

z-score here is meant to be the weighted average of z-scores of each individual metric (as described in sect 5.2 of the Dynaboard paper)
averaging is just an average of raw scores with all metrics and datasets weighed equally
perf-only is just 100% performance metric, discard throughput/memory/deltametrics

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.