You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Nov 1, 2024. It is now read-only.
Process the new taskConfig.ATTRIBUTE_NAME attribute accordingly with a default value of min (not sure if we need to make any other changes to make it editable)
Change .getRandomContext(this.state.task.id, this.state.task.cur_round) to .getRandomContext(this.state.task.id, this.state.task.cur_round, [], taskConfig.ATTRIBUTE_NAME)
The text was updated successfully, but these errors were encountered:
Might be easier to control it with a new place in the task owner settings (similar to how the validation threshold is selected). Do you have any objections to doing that instead?
The current default method for getRandomContext() is
min
, however, there is no way for task owners to change this currently.Ideally this should be controlled from the task config (e.g. https://dynabench.org/task-owner-interface/qa#advanced) similar to
aggregation_metric
.We would need to modify: https://github.com/facebookresearch/dynabench/blob/main/frontends/web/src/common/Annotation/CreateInterface.js#L95.
I suspect the steps required to implement are:
taskConfig
loading slightly earliertaskConfig.ATTRIBUTE_NAME
attribute accordingly with a default value ofmin
(not sure if we need to make any other changes to make it editable).getRandomContext(this.state.task.id, this.state.task.cur_round)
to.getRandomContext(this.state.task.id, this.state.task.cur_round, [], taskConfig.ATTRIBUTE_NAME)
The text was updated successfully, but these errors were encountered: