Skip to content

Issues: stanford-crfm/helm

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

125 Open 961 Closed
125 Open 961 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Support few-shot chain-of-thought in GPQA / MMLU
#3088 opened Oct 22, 2024 by yifanmai
Issue with running HEIM documentation Improvements or additions to documentation HEIM (Text2Image) user question
#3080 opened Oct 22, 2024 by sudhir-mcw
2
flake8 in Python 3.12 produces a large number of errors
#3072 opened Oct 18, 2024 by yifanmai
Optimum Intel OpenVino fails with segmentation fault bug Something isn't working models
#3066 opened Oct 16, 2024 by yifanmai
How to use this package when I have the pompts and images across models HEIM (Text2Image) user question
#3062 opened Oct 14, 2024 by snehith3195
1
Not able to install install-heim-extras.sh for heim leaderboard HEIM (Text2Image) user question
#3060 opened Oct 11, 2024 by snehith3195
1
Proposal: Multi-turn support
#3059 opened Oct 10, 2024 by yifanmai
Proposal: User-provided custom adapters
#3058 opened Oct 10, 2024 by yifanmai
Proposal: Add free-form data fields to Instance
#3057 opened Oct 10, 2024 by yifanmai
1
HatefulMemesScenario get_instances returning error bug Something isn't working HEIM (Text2Image) methodology Evaluation methodology user question
#3056 opened Oct 10, 2024 by dxwu2
3
Add new model (upstage/solar-pro-preview-instruct)
#3040 opened Oct 7, 2024 by remy-rec
1
Add MMLU-Pro additions New models or scenarios good first issue Good for newcomers scenarios
#3018 opened Sep 24, 2024 by yifanmai
@siyagoel
Help me understand the HELM Classic Leaderboard's missing results user question
#2994 opened Sep 16, 2024 by PaulJoeMaliakel
1
Add metric tooltip to Predictions
#2987 opened Sep 11, 2024 by farzaank
Add amazon bedrock 3p models to the model configuration. user question
#2967 opened Sep 3, 2024 by subhaviv
2
Incorrect scoring due to answer format mismatch in MMLU evaluation user question
#2939 opened Aug 16, 2024 by DerryChan
1
Add wrapper with consistent padding for MiniLeaderboards
#2933 opened Aug 15, 2024 by farzaank
@farzaank
Breaking change in nltk 3.8.2
#2926 opened Aug 14, 2024 by yifanmai
RAFT evaluation user question
#2909 opened Aug 8, 2024 by divyashan
2
Add Windows support
#2907 opened Aug 8, 2024 by yifanmai
Add sorting/filtering functionality to predictions pages frontend Frontend issues
#2887 opened Aug 1, 2024 by farzaank
Add metadata field to HELM predictions enhancement New feature or request frontend Frontend issues
#2886 opened Aug 1, 2024 by farzaank
2
VHELM images do not scale when window is resized
#2885 opened Aug 1, 2024 by yifanmai
HEIMHumanEvalScenario requires permissions to download data from codalab HEIM (Text2Image) user question
#2865 opened Jul 30, 2024 by slymane
1
pip needs to be temporarily downgraded to 24.1.2
#2855 opened Jul 30, 2024 by yifanmai
ProTip! What’s not been updated in a month: updated:<2024-09-24.