MLCommons

All

92 repositories

modelbench
Public
Run safety benchmarks against AI models and view detailed reports showing how well they performed.
Python
•
Apache License 2.0
•8•58•211•3•Updated Oct 26, 2024Oct 26, 2024
ck
Public
Collective Knowledge (CK) is an educational community project to learn how to run AI, ML and other emerging workloads in the most efficient and cost-effective way across diverse models, data sets, software and hardware using MLCommons CM (Collective Mind workflow automation framework)
education automation optimization best-practices collaboration modularity tournaments portability workflow-automation reusability
Python
•
Apache License 2.0
•113•606•14•1•Updated Oct 25, 2024Oct 25, 2024
cm4mlperf-inference
Public
Apache License 2.0
•24•0•4•2•Updated Oct 25, 2024Oct 25, 2024
inference
Public
Reference implementations of MLPerf™ inference benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•532•1.2k•162•23•Updated Oct 25, 2024Oct 25, 2024
cm4mlops
Public
A collection of portable, reusable and cross-platform automation recipes (CM scripts) to make it easier to build and benchmark AI systems across diverse models, data sets, software and hardware
workflow devops machine-learning automation cross-platform reusable workflow-automation mlops mlperf
Python
•
Apache License 2.0
•17•10•81•1•Updated Oct 25, 2024Oct 25, 2024
GaNDLF-Synth
Public
Python
•
Apache License 2.0
•1•3•0•1•Updated Oct 25, 2024Oct 25, 2024
dynabench
Public
Python
•
MIT License
•16•18•7•4•Updated Oct 24, 2024Oct 24, 2024
policies
Public
General policies for MLPerf™ including submission rules, coding standards, etc.
Python
•
Apache License 2.0
•54•27•35•40•Updated Oct 24, 2024Oct 24, 2024
mobile_app_open
Public
Mobile App Open
C++
•
Apache License 2.0
•22•46•30•4•Updated Oct 24, 2024Oct 24, 2024
chakra
Public
Repository for MLCommons Chakra schema and tools
Python
•
Apache License 2.0
•35•62•20•9•Updated Oct 23, 2024Oct 23, 2024
croissant
Public
Croissant is a high-level format for machine learning datasets that brings together four rich layers.
machine-learning schema-org json-ld datasets
Python
•
Apache License 2.0
•40•434•112•17•Updated Oct 23, 2024Oct 23, 2024
GaNDLF
Public
A generalizable application framework for segmentation, regression, and classification using PyTorch
framework regression medical-imaging classification segmentation data-augmentation biomedical-image-processing medical-informatics medical-image-analysis clinical-workflow
Python
•
Apache License 2.0
•79•161•20•3•Updated Oct 22, 2024Oct 22, 2024
inference_policies
Public
Issues related to MLPerf™ Inference policies, including rules and suggested changes
Apache License 2.0
•52•56•18•5•Updated Oct 22, 2024Oct 22, 2024
algorithmic-efficiency
Public
MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
Python
•
Apache License 2.0
•64•327•27•7•Updated Oct 22, 2024Oct 22, 2024
training
Public
Reference implementations of MLPerf™ training benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•556•1.6k•17•42•Updated Oct 17, 2024Oct 17, 2024
logging
Public
MLPerf™ logging library
Python
•
Apache License 2.0
•46•30•40•1•Updated Oct 17, 2024Oct 17, 2024
inference_results_v4.0
Public
This repository contains the results and code for the MLPerf™ Inference v4.0 benchmark.
Apache License 2.0
•16•10•5•4•Updated Oct 17, 2024Oct 17, 2024
inference_results_v4.1
Public
This repository contains the results and code for the MLPerf™ Inference v4.1 benchmark.
benchmark machine-learning
Python
•
Apache License 2.0
•7•2•4•1•Updated Oct 16, 2024Oct 16, 2024
mobile_results_v4.1
Public
This repository contains the results and code for the MLPerf™ Mobile Inference v4.1 benchmark.
Apache License 2.0
•1•0•0•0•Updated Oct 16, 2024Oct 16, 2024
inference_results_v3.1
Public
This repository contains the results and code for the MLPerf™ Inference v3.1 benchmark.
Apache License 2.0
•13•11•9•2•Updated Oct 16, 2024Oct 16, 2024
tiny
Public
MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers
C
•
Apache License 2.0
•85•354•13•4•Updated Oct 16, 2024Oct 16, 2024
mobile_models
Public
MLPerf™ Mobile models
Apache License 2.0
•10•24•4•0•Updated Oct 16, 2024Oct 16, 2024
mobile_open
Public
MLPerf Mobile benchmarks
Python
•
Apache License 2.0
•2•10•6•2•Updated Oct 16, 2024Oct 16, 2024
medperf
Public
An open benchmarking platform for medical artificial intelligence using Federated Evaluation.
Python
•
Apache License 2.0
•30•144•51•21•Updated Oct 15, 2024Oct 15, 2024
algorithms_results_v0.5
Public
This repository contains the results and code for the AlgoPerf v0.5 benchmark.
benchmark algorithms
Python
•
Apache License 2.0
•0•3•0•1•Updated Oct 4, 2024Oct 4, 2024
training_policies
Public
Issues related to MLPerf™ training policies, including rules and suggested changes
Python
•
Apache License 2.0
•66•92•109•3•Updated Oct 3, 2024Oct 3, 2024
modelgauge
Public archive
Make it easy to automatically and uniformly measure the behavior of many AI Systems.
Python
•
Apache License 2.0
•7•26•63•0•Updated Oct 2, 2024Oct 2, 2024
inference_results_visualization_template
Public template
JavaScript
•
Apache License 2.0
•3•0•1•1•Updated Oct 1, 2024Oct 1, 2024
storage_results_v1.0
Public
This repository contains the results and code for the MLPerf™ Storage v1.0 benchmark.
Python
•
Apache License 2.0
•0•2•0•0•Updated Sep 25, 2024Sep 25, 2024
submissions_algorithms
Public
Apache License 2.0
•0•1•1•0•Updated Sep 23, 2024Sep 23, 2024