BigScience Workshop

All

36 repositories

data_tooling
Public
Tools for managing datasets for governance and training.
HTML
•
Apache License 2.0
•48•77•138•3•Updated Oct 28, 2024Oct 28, 2024
biomedical
Public
Tools for curating biomedical training data for large-scale language modeling
Python
•115•459•165•18•Updated Oct 26, 2024Oct 26, 2024
ShadesofBias
Public
Evaluation for Shades of Bias in Text
HTML
•0•0•1•0•Updated Oct 21, 2024Oct 21, 2024
xmtf
Public
Crosslingual Generalization through Multitask Finetuning
bloom language-models zero-shot-learning multitask-learning t5 multilingual-nlp large-language-models instruction-tuning bloomz mt0
Jupyter Notebook
•
Apache License 2.0
•37•515•11•0•Updated Sep 22, 2024Sep 22, 2024
petals
Public
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
nlp bloom distributed-systems machine-learning deep-learning chatbot pytorch falcon transformer neural-networks
Python
•
MIT License
•522•9.2k•87•18•Updated Sep 7, 2024Sep 7, 2024
bigscience
Public
Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
nlp training machine-learning models
Shell
•
Other
•103•979•13•8•Updated Jul 29, 2024Jul 29, 2024
Megatron-DeepSpeed
Public
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Python
•
Other
•215•1.3k•74•45•Updated Mar 20, 2024Mar 20, 2024
multilingual-modeling
Public
BLOOM+1: Adapting BLOOM model to support a new unseen language
Python
•
Apache License 2.0
•16•70•13•6•Updated Mar 2, 2024Mar 2, 2024
promptsource
Public
Toolkit for creating, sharing and using natural language prompts.
nlp machine-learning natural-language-processing
Python
•
Apache License 2.0
•352•2.7k•11•32•Updated Oct 23, 2023Oct 23, 2023
massive-probing-framework
Public
Framework for BLOOM probing
Python
•10•8•0•0•Updated Oct 17, 2023Oct 17, 2023
architecture-objective
Public
Python
•
Apache License 2.0
•307•93•4•5•Updated Jul 25, 2023Jul 25, 2023
metadata
Public
Experiments on including metadata such as URLs, timestamps, website descriptions and HTML tags during pretraining.
Python
•
Apache License 2.0
•12•30•25•13•Updated Jun 12, 2023Jun 12, 2023
lm-evaluation-harness
Public
A framework for few-shot evaluation of autoregressive language models.
Python
•
MIT License
•1.8k•101•7•8•Updated May 9, 2023May 9, 2023
data-preparation
Public
Code used for sourcing and cleaning the BigScience ROOTS corpus
multilingual large-language-models dataset
Jupyter Notebook
•
Apache License 2.0
•40•305•9•0•Updated Mar 20, 2023Mar 20, 2023
bibliography
Public
A list of BigScience publications
research bibtex bibliography
TeX
•
Apache License 2.0
•1•3•1•0•Updated Mar 13, 2023Mar 13, 2023
multilingual-modeling-1
Public
Python
•
Apache License 2.0
•16•2•0•0•Updated Dec 5, 2022Dec 5, 2022
t-zero
Public
Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)
Python
•
Apache License 2.0
•53•457•7•2•Updated Nov 5, 2022Nov 5, 2022
carbon-footprint
Public
A repository for `codecarbon` logs.
Jupyter Notebook
•6•10•1•0•Updated Nov 3, 2022Nov 3, 2022
pii_processing
Public
PII Processing code to detect and remediate PII in BigScience datasets. Reference implementation for the PII Hackathon
Python
•
Other
•6•8•7•1•Updated Oct 6, 2022Oct 6, 2022
lam
Public
Libraries, Archives and Museums (LAM)
Apache License 2.0
•7•82•34•0•Updated Oct 4, 2022Oct 4, 2022
model_card
Public
Apache License 2.0
•4•24•0•0•Updated Jul 11, 2022Jul 11, 2022
bloom-dechonk
Public
A repo for running model shrinking experiments
Python
•4•10•0•0•Updated Jun 21, 2022Jun 21, 2022
historical_texts
Public
BigScience working group on language models for historical texts
Jupyter Notebook
•6•8•0•1•Updated May 10, 2022May 10, 2022
evaluation
Public
Code and Data for Evaluation WG
Python
•
Other
•24•41•41•9•Updated May 4, 2022May 4, 2022
catalogue_data
Public
Scripts to prepare catalogue data
Jupyter Notebook
•
Apache License 2.0
•1•8•5•3•Updated Apr 25, 2022Apr 25, 2022
training_dynamics
Public
4•5•11•2•Updated Apr 22, 2022Apr 22, 2022
evaluation-robustness-consistency
Public
Tools for evaluating model robustness and consistency
Python
•
Other
•2•2•0•2•Updated Mar 9, 2022Mar 9, 2022
interpretability-ideas
Public
1•1•10•0•Updated Feb 27, 2022Feb 27, 2022
tokenization
Public
Python
•
Apache License 2.0
•2•11•1•2•Updated Feb 16, 2022Feb 16, 2022
datasets_stats
Public
Generate statistics over datasets used in the context of BS
Makefile
•1•2•0•0•Updated Feb 1, 2022Feb 1, 2022