The evaluation of MBPP is 1-shot or 3-shot？ #867
Unanswered
xiaoshengjun
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
In https://rank.opencompass.org.cn/leaderboard-llm shows the evaluation of MBPP is 1-shot， When the mouse is placed on the test score of MBPP。But in code configs/datasets/mbpp/mbpp_gen_1e1056.py, 'mbpp_infer_cfg' config file shows 3-shot. So 1-shot or 3-shot, which is used for MBPP?
Beta Was this translation helpful? Give feedback.
All reactions