Skip to content

@epfml EPFL Machine Learning and Optimization Laboratory

Change the repository type filter

All

    Repositories list

    59 repositories

    • disco

      Public
      DISCO is a code-free and installation-free browser platform that allows any non-technical user to collaboratively train machine learning models without sharing any private data.
      machine-learningmobilebrowsercollaborative-learningprivacy-preservingfederated-learningdecentralized-learning
      TypeScript
      •
      Apache License 2.0
      •26•158•55•12•Updated Dec 23, 2024Dec 23, 2024

    • llm-baselines

      Public
      nanoGPT-like codebase for LLM training
      pretrainingllms
      Python
      •
      MIT License
      •24•81•3•4•Updated Dec 18, 2024Dec 18, 2024

    • ML_course

      Public
      EPFL Machine Learning Course, Fall 2024
      Jupyter Notebook
      •916•1.3k•3•1•Updated Dec 10, 2024Dec 10, 2024

    • prefixlm

      Public
      Python
      •
      MIT License
      •0•1•0•0•Updated Dec 8, 2024Dec 8, 2024

    • getting-started

      Public
      Python
      •11•18•0•3•Updated Nov 5, 2024Nov 5, 2024

    • schedules-and-scaling

      Public
      Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"
      Python
      •
      MIT License
      •2•65•0•0•Updated Oct 30, 2024Oct 30, 2024

    • powersgd

      Public
      Practical low-rank gradient compression for distributed optimization: https://arxiv.org/abs/1905.13727
      Python
      •
      MIT License
      •33•144•1•1•Updated Oct 29, 2024Oct 29, 2024

    • CoBo

      Public
      Python
      •0•0•0•0•Updated Oct 22, 2024Oct 22, 2024

    • CoMiGS

      Public
      Python
      •
      MIT License
      •0•0•0•0•Updated Oct 2, 2024Oct 2, 2024

    • personalized-collaborative-llms

      Public
      Exploration on-device self-supervised collaborative fine-tuning of large language models with limited local data availability, using Low-Rank Adaptation (LoRA). We introduce three distinct trust-weighted gradient aggregation schemes: weight similarity-based, prediction similarity-based and validation performance-based.
      Python
      •
      Apache License 2.0
      •0•3•1•0•Updated Sep 2, 2024Sep 2, 2024

    • error-feedback-SGD

      Public
      SGD with compressed gradients and error-feedback: https://arxiv.org/abs/1901.09847
      Jupyter Notebook
      •
      MIT License
      •10•30•2•2•Updated Jul 25, 2024Jul 25, 2024

    • OptML_course

      Public
      EPFL Course - Optimization for Machine Learning - CS-439
      Jupyter Notebook
      •320•1.2k•5•0•Updated Jun 27, 2024Jun 27, 2024

    • REQ

      Public
      Python
      •
      Apache License 2.0
      •0•16•0•0•Updated Jun 10, 2024Jun 10, 2024

    • CoTFormer

      Public
      Python
      •
      MIT License
      •0•0•0•0•Updated May 22, 2024May 22, 2024

    • semester-project-personalization

      Public
      Python
      •0•0•0•0•Updated May 22, 2024May 22, 2024

    • getting-started-lauzhack

      Public
      Python
      •11•0•0•0•Updated Apr 18, 2024Apr 18, 2024

    • DenseFormer

      Public
      Python
      •
      Apache License 2.0
      •8•78•0•1•Updated Apr 16, 2024Apr 16, 2024

    • DoGE

      Public
      Codebase for ICML submission "DOGE: Domain Reweighting with Generalization Estimation"
      4•0•0•0•Updated Feb 4, 2024Feb 4, 2024

    • landmark-attention

      Public
      Landmark Attention: Random-Access Infinite Context Length for Transformers
      Python
      •
      Apache License 2.0
      •36•418•8•1•Updated Dec 20, 2023Dec 20, 2023

    • pam

      Public
      Python
      •
      Apache License 2.0
      •3•14•0•0•Updated Dec 9, 2023Dec 9, 2023

    • ghost-noise

      Public
      Python
      •
      Apache License 2.0
      •0•4•0•0•Updated Aug 18, 2023Aug 18, 2023

    • optML-pku

      Public
      summer school materials
      5•44•0•0•Updated Aug 4, 2023Aug 4, 2023

    • collaborative-attention

      Public
      Code for Multi-Head Attention: Collaborate Instead of Concatenate
      Python
      •
      Apache License 2.0
      •22•150•5•1•Updated Jun 12, 2023Jun 12, 2023

    • dynamic-sparse-flash-attention

      Public
      Jupyter Notebook
      •
      Other
      •6•133•2•0•Updated Jun 2, 2023Jun 2, 2023

    • easy-summary

      Public
      difficulty-guided text summarization
      Python
      •
      Apache License 2.0
      •5•5•0•0•Updated May 22, 2023May 22, 2023

    • relaysgd

      Public
      Code for the paper “RelaySum for Decentralized Deep Learning on Heterogeneous Data”
      Jupyter Notebook
      •
      MIT License
      •2•9•0•0•Updated Apr 21, 2023Apr 21, 2023

    • epfml-utils

      Public
      Tools for experimentation and using run:ai. The aim is for these to be small self-contained utilities that are used by multiple people.
      Python
      •
      Apache License 2.0
      •0•0•1•0•Updated Mar 16, 2023Mar 16, 2023

    • cifar

      Public
      MLO internal cifar 10 / 100 default implementation / reference implementation. single machine, variable batch sizes, allowing maybe gradient compression. need to have clear documentation to make it easy to use, and so that we don't loose time with looking for hyperparameters. we can later keep it in sync with mlbench too, but self-contained is e…
      Python
      •0•0•0•1•Updated Feb 8, 2023Feb 8, 2023

    • attention-cnn

      Public
      Source code for "On the Relationship between Self-Attention and Convolutional Layers"
      Python
      •
      Apache License 2.0
      •127•1.1k•6•0•Updated Jan 10, 2023Jan 10, 2023

    • federated-learning-public-code

      Public
      Python
      •47•156•3•1•Updated Dec 23, 2022Dec 23, 2022