You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If I want to eval on MMLU, what is the proper value of the param max_new_tokens? The default value 1024 is too big, and the time consumes a lot. I change it to 1, but the prediction is almost /n, just a few ABCD answers.
How to do the tradeoff between the time and the accuracy?
My yaml config is as follows:
from mmengine.config import read_base
from opencompass.models import LightllmAPI
from opencompass.partitioners import NaivePartitioner
from opencompass.runners import LocalRunner
from opencompass.tasks import OpenICLInferTask
with read_base():
# from .datasets.humaneval.humaneval_gen import humaneval_datasets
from .datasets.mmlu.mmlu_gen import mmlu_datasets
datasets = [*mmlu_datasets]
models = [
dict(
abbr='LightllmAPI',
type=LightllmAPI,
url='http://localhost:8080/generate',
input_format='<input_text_to_replace>',
max_seq_len=2048,
batch_size=32,
generation_kwargs=dict(
do_sample=False,
ignore_eos=False,
max_new_tokens=1
),
),
]
infer = dict(
partitioner=dict(type=NaivePartitioner),
runner=dict(
type=LocalRunner,
max_num_workers=32,
task=dict(type=OpenICLInferTask),
),
)
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
If I want to eval on MMLU, what is the proper value of the param
max_new_tokens
? The default value1024
is too big, and the time consumes a lot. I change it to1
, but the prediction is almost/n
, just a fewABCD
answers.How to do the tradeoff between the time and the accuracy?
My yaml config is as follows:
Beta Was this translation helpful? Give feedback.
All reactions