update website to move mlrc2023 content to its own folders

config/_default/config.yaml

@@ -3,7 +3,7 @@
 # Hugo Documentation: https://gohugo.io/getting-started/configuration/#all-configuration-settings
 # This file is formatted using YAML syntax - learn more at https://learnxinyminutes.com/docs/yaml/
 
-title: 'MLRC2023' # Website name
+title: 'MLRC' # Website name
 baseURL: 'https://example.com/' # Website URL
 
 ############################

config/_default/menus.yaml

@@ -14,6 +14,10 @@ main:
     identifier: past
     url : #
     weight: 30
+  - name: MLRC 2023
+    url: /proceedings/mlrc2023/
+    weight: 5
+    parent: past
   - name: MLRC 2022
     url: https://paperswithcode.com/rc2022
     weight: 10

content/_index.md

@@ -1,179 +1,39 @@
 ---
-title: ML Reproducibility Challenge 2023
+title: ML Reproducibility Challenge
 type: book # Do not modify.
 toc: false
 headless: true
 ---
 
-Welcome to the ML Reproducibility Challenge 2023 (**MLRC 2023**). This is the
-seventh edition of the event
+Welcome to the home of ML Reproducibility Challenge. This is an annual event for
+providing a space for research into reproducibility of Machine Learning
+literature.
 ([v1](https://www.cs.mcgill.ca/~jpineau/ICLR2018-ReproducibilityChallenge.html),
 [v2](https://www.cs.mcgill.ca/~jpineau/ICLR2019-ReproducibilityChallenge.html),
 [v3](https://reproducibility-challenge.github.io/neurips2019/),
 [v4](https://paperswithcode.com/rc2020),
 [v5](https://paperswithcode.com/rc2021),
-[v6](https://paperswithcode.com/rc2022)). The primary goal of this event is to
-encourage the publishing and sharing of scientific results that are reliable and
-reproducible. In support of this, the objective of this challenge is to
-investigate reproducibility of papers accepted for publication at top
-conferences by inviting members of the community at large to select a paper, and
-verify the empirical results and claims in the paper by reproducing the
-computational experiments, either via a new implementation or using code/data or
-other information provided by the authors.
-
-## Final decisions for MLRC 2023
-
-We are now releasing the final list of decisions for MLRC 2023. This list
-includes the previous partial list published on July 5th, 2024. We have given
-additional time to TMLR to complete the reviews, however it is unfortunate that
-few papers are still awaiting a decision due to unresponsive Action Editors from
-TMLR. As we need to wrap up this edition, we are proceeding with the final list
-of 22 accepted papers. Congratulations to all!
-
-- Ana-Maria Vasilcoiu, Batu Helvacioğlu, Thies Kersten, Thijs Stessen;
-  _GNNInterpreter: A probabilistic generative model-level explanation for Graph
-  Neural Networks_, [OpenReview](https://openreview.net/forum?id=8cYcR23WUo)
-- Miklos Hamar, Matey Krastev, Kristiyan Hristov, David Beglou; _Explaining
-  Temporal Graph Models through an Explorer-Navigator Framework_,
-  [OpenReview](https://openreview.net/forum?id=FI1XvwpchC)
-- Clio Feng, Colin Bot, Bart den Boef, Bart Aaldering; _Reproducibility Study of
-  "Explaining RL Decisions with Trajectories"_,
-  [OpenReview](https://openreview.net/forum?id=JQoWmeNaC2)
-- Ethan Harvey, Mikhail Petrov, Michael C. Hughes; _Transfer Learning with
-  Informative Priors: Simple Baselines Better than Previously Reported_,
-  [OpenReview](https://openreview.net/forum?id=BbvSU02jLg)
-- Gijs de Jong,Macha Meijer,Derck W.E. Prinzhorn,Harold Ruiter; _Reproducibility
-  study of FairAC_, [OpenReview](https://openreview.net/forum?id=ccDi5jtSF7)
-- Nesta Midavaine, Gregory Hok Tjoan Go, Diego Canez, Ioana Simion, Satchit
-  Chatterji; _On the Reproducibility of Post-Hoc Concept Bottleneck Models_;
-  [OpenReview](https://openreview.net/forum?id=8UfhCZjOV7)
-- Jiapeng Fan, Paulius Skaigiris, Luke Cadigan, Sebastian Uriel Arias;
-  _Reproducibility Study of "Learning Perturbations to Explain Time Series
-  Predictions"_, [OpenReview](https://openreview.net/forum?id=fCNqD2IuoD)
-- Karim Ahmed Abdel Sadek, Matteo Nulli, Joan Velja, Jort Vincenti; _Explaining
-  RL Decisions with Trajectories’: A Reproducibility Study_,
-  [OpenReview](https://openreview.net/forum?id=QdeBbK5CSh)
-- Markus Semmler, Miguel de Benito Delgado; _Classwise-Shapley values for data
-  valuation_ [OpenReview](https://openreview.net/forum?id=srFEYJkqD7)
-- Daniel Gallo Fernández, Răzvan-Andrei Matișan, Alejandro Monroy Muñoz, Janusz
-  Partyka; _Reproducibility Study of "ITI-GEN: Inclusive Text-to-Image
-  Generation"_ [OpenReview](https://openreview.net/forum?id=d3Vj360Wi2)
-- Kacper Bartosik, Eren Kocadag, Vincent Loos, Lucas Ponticelli;
-  _Reproducibility study of "Robust Fair Clustering: A Novel Fairness Attack and
-  Defense Framework"_, [OpenReview](https://openreview.net/forum?id=Xu1sEPhjqH)
-- Barath Chandran C; _CUDA: Curriculum of Data Augmentation for Long‐Tailed
-  Recognition_, [OpenReview](https://openreview.net/forum?id=Wm6d44I8St)
-- Christina Isaicu, Jesse Wonnink, Andreas Berentzen, Helia Ghasemi;
-  _Reproducibility Study of “Explaining Temporal Graph Models Through an
-  Explorer-Navigator Framework"_,
-  [OpenReview](https://openreview.net/forum?id=9M2XqvH2SB)
-- Iason Skylitsis, Zheng Feng, Idries Nasim, Camille Niessink; _Reproducibility
-  Study of "Robust Fair Clustering: A Novel Fairness Attack and Defense
-  Framework"_, [OpenReview](https://openreview.net/forum?id=H1hLNjwrGy)
-- Fatemeh Nourilenjan Nokabadi, Jean-Francois Lalonde, Christian Gagné;
-  _Reproducibility Study on Adversarial Attacks Against Robust Transformer
-  Trackers_, [OpenReview](https://openreview.net/forum?id=FEEKR0Vl9s)
-- Luan Fletcher, Robert van der Klis, Martin Sedlacek, Stefan Vasilev, Christos
-  Athanasiadis; _Reproducibility study of “LICO: Explainable Models with
-  Language-Image Consistency"_,
-  [OpenReview](https://openreview.net/forum?id=Mf1H8X5DVb)
-- Wouter Bant, Ádám Divák, Jasper Eppink, Floris Six Dijkstra; _On the
-  Reproducibility of: "Learning Perturbations to Explain Time Series
-  Predictions"_, [OpenReview](https://openreview.net/forum?id=nPZgtpfgIx)
-- Berkay Chakar,Amina Izbassar,Mina Janićijević,Jakub Tomaszewski;
-  _Reproducibility Study: Equal Improvability: A New Fairness Notion Considering
-  the Long-Term Impact_,
-  [OpenReview](https://openreview.net/forum?id=Yj8fUQGXXL)
-- Oliver Bentham, Nathan Stringham, Ana Marasović; _Chain-of-Thought
-  Unfaithfulness as Disguised Accuracy_,
-  [OpenReview](https://openreview.net/forum?id=ydcrP55u2e)
-- Shivank Garg, Manyana Tiwari; _Unmasking the Veil: An Investigation into
-  Concept Ablation for Privacy and Copyright Protection in Images_
-  [OpenReview](https://openreview.net/forum?id=TYYApLzjaQ)
-- Adrian Sauter, Milan Miletić, Ryan Ott, Rohith Saai Pemmasani Prabakaran;
-  _Studying How to Efficiently and Effectively Guide Models with Explanations” -
-  A Reproducibility Study_,
-  [OpenReview](https://openreview.net/forum?id=9ZzASCVhDF)
-- Thijmen Nijdam, Taiki Papandreou-Lazos, Jurgen de Heus, Juell Sprott;
-  _Reproducibility Study Of Learning Fair Graph Representations Via Automated
-  Data Augmentations_, [OpenReview](https://openreview.net/forum?id=4WiqHopXQX)
-
-If you are an author of the below mentioned papers and have not
-[submitted the form](https://forms.gle/JJ28rLwBSxMriyE89) with the camera ready
-items, please consider doing so at the earliest. We will reach out to the
-accepted authors soon with the next steps. We will also announce the best paper
-awards and share details on the logistics of NeurIPS poster session in the
-coming weeks.
-
-**Update, Sept 13th, 2024**: A couple of papers received acceptance status post
-our final date of MLRC 2023 acceptance. We have now incorporated them too in the
-final list.
-
-## An update on decisions
-
-_July 5th, 2024_
-
-We initially communicated to have all decisions of MLRC 2023 out by 31st of
-May, 2024. Unfortunately, several submissions are still under review at TMLR,
-and we are waiting for the final decisions to trickle in. Overall, MLRC 2023 had
-46 valid submissions, out of which we have recieved decisions on 61% of them. We
-are in touch with TMLR to expedite the process of decisions for the remaining
-submissions - we expect all decisions to come in by the next couple of weeks.
-
-Until then, we are happy to announce the (partial) list of accepted papers.
-Congratulations to all :tada:! If you are an author of the below mentioned
-papers and have not [submitted the form](https://forms.gle/JJ28rLwBSxMriyE89)
-with the camera ready items, please consider doing so at the earliest. We will
-reach out to the accepted authors soon with the next steps.
-
-_(partial paper list removed as we release the final list above)_
-
-## [Deprecated] Call For Papers
-
-We invite contributions from academics, practitioners and industry researchers
-of the ML community to submit novel and insightful reproducibility studies.
-Please read our [blog post](/blog/announcing_mlrc2023/) regarding our
-retrospectives of running the challenge and the future roadmap. We are happy to
-announce the formal partnership with
-[Transactions of Machine Learning Research (TMLR)](https://jmlr.org/tmlr/)
-journal. The challenge goes live on **October 23, 2023**.
-
-We recommend you choose any paper(s) published in the 2023 calendar year from
-the top conferences and journals ([NeurIPS](https://neurips.cc/),
-[ICML](https://icml.cc/), [ICLR](https://iclr.cc/),
-[ACL](https://2023.aclweb.org/), [EMNLP](https://2023.emnlp.org/),
-[ICCV](https://iccv2023.thecvf.com/),
-[CVPR](https://cvpr2023.thecvf.com/Conferences/2023),
-[TMLR](https://jmlr.org/tmlr/), [JMLR](https://jmlr.org/),
-[TACL](https://transacl.org/index.php/tacl)) to run your reproducibility study
-on.
-
-{{< figure src="uploads/mlrc.drawio.svg" class="mlrc_dark" >}}
-
-{{< figure src="uploads/mlrc.light.drawio.svg" class="mlrc_light" >}}
-
-In order for your paper to be submitted and presented at MLRC 2023, it first
-needs to be **accepted and published** at TMLR. While TMLR aims to follow a
-2-months timeline to complete the review process of its regular submissions,
-this timeline is not guaranteed. If you haven’t already, we therefore recommend
-submitting your original paper to TMLR by **February 16th, 2024**, that is a
-little over 3 months in advance of the MLRC publication announcement date.
-
-## Key Dates
-
-- Challenge goes live: October 23, 2023
-- Deadline to share your **intent to submit** a TMLR paper to MLRC: **February
-  16th, 2024** at the following form: **https://forms.gle/JJ28rLwBSxMriyE89**.
-  This form requires that you provide a link to your TMLR submission. Once it
-  gets accepted (if it isn’t already), you should then update the same form with
-  your paper camera ready details. Your accepted TMLR paper will finally undergo
-  a light AC review to verify MLRC compatibility.
-- We aim to announce the accepted papers by ~~**May 31st, 2024**~~ **July 17th,
-  2024**, pending decisions of all papers.
-
-## Contact Information
-
-- For query regarding MLRC 2023, mail us at:
-  [[email protected]](mailto:[email protected]).
-- For general queries, media, sponsorship, partnership requests, mail us at
-  [[email protected]]([email protected]).
+[v6](https://paperswithcode.com/rc2022), [v7](/proceedings/mlrc2023/)). The
+primary goal of this event is to encourage the publishing and sharing of
+scientific results that are reliable and reproducible. In support of this, the
+objective of this challenge is to investigate reproducibility of papers accepted
+for publication at top conferences by inviting members of the community at large
+to select a paper, and verify the empirical results and claims in the paper by
+reproducing the computational experiments, either via a new implementation or
+using code/data or other information provided by the authors.
+
+{{% callout note %}}
+
+- :mortar_board: [MLRC 2023](/proceedings/mlrc2023/) papers featured in
+  [NeurIPS 2024 Poster Sessions](https://neurips.cc/), Dec 10-15, 2024 at
+  Vancouver, Canada. If you are attending NeurIPS, do
+  [drop by to the posters](/proceedings/) to say hi!
+- Next iteration of MLRC will be **MLRC2025**, and it will be **in-person** - a
+  one-day conference! Announcement will be made very soon, stay tuned!
+
+{{% /callout %}}
+
+{{< tweet user="hugo_larochelle" id="1819465878641262862" >}}
+
+<a href="https://twitter.com/x?ref_src=twsrc%5Etfw" class="twitter-follow-button" data-show-count="false">Follow
+@x</a><script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

content/call_for_papers/_index.md

@@ -7,68 +7,70 @@ icon_pack: fas
 
 # Page metadata.
 title: Call for Papers
-date: "2023-10-22T00:00:00Z"
+date: "2024-12-10T00:00:00Z"
 type: book # Do not modify.
 ---
 
-The Machine Learning Reproducibility Challenge (MLRC 2023) is an unique, online
-conference which encourages the community to investigate the reproducibility,
-replicability and generalisability of published claims in top conferences in the
-literature. We invite submissions which investigate the recently published
-claims, add novel insights to them, and enable reproducible research, spanning
-various topics in the ML literature. Submissions must be first accepted at
-[TMLR](https://jmlr.org/tmlr/) to be considered in the MLRC 2023 Proceedings.
-Please read the [author guidelines](https://jmlr.org/tmlr/author-guide.html) and
-[submission guidelines](https://jmlr.org/tmlr/editorial-policies.html) from TMLR
-to get the submission format and to understand more about the reviewing process.
-Please read our [announcement blog post](/blog/announcing_mlrc2023/) for more
-motivation, retrospectives and roadmap for the challenge.
-
-## Scope
-
-We invite thorough reproducibility studies, including but not limited to:
-
-- _Generalisability_ of published claims: novel insights and results beyond what
-  was presented in the original paper. We recommend you choose any paper(s)
-  published in the 2023 calendar year from the top conferences and journals
-  ([NeurIPS](https://neurips.cc/), [ICML](https://icml.cc/),
-  [ICLR](https://iclr.cc/), [ACL](https://2023.aclweb.org/),
-  [EMNLP](https://2023.emnlp.org/), [ICCV](https://iccv2023.thecvf.com/),
-  [CVPR](https://cvpr2023.thecvf.com/Conferences/2023),
-  [TMLR](https://jmlr.org/tmlr/), [JMLR](https://jmlr.org/),
-  [TACL](https://transacl.org/index.php/tacl)) to run your reproducibility study
-  on.
-- Meta-reproducibility studies on set of related papers.
-- Research on tools enabling reproducible research.
-- Meta analysis on the state of reproducibility in various subfields in Machine
-  Learning.
-
-## Important Dates
-
-- Challenge goes live: October 23, 2023
-- Submit to TMLR OpenReview: https://openreview.net/group?id=TMLR
-- Deadline to share your **intent to submit** a TMLR paper to MLRC: **February
-  16th, 2024** at the following form: **https://forms.gle/JJ28rLwBSxMriyE89**.
-  This form requires that you provide a link to your TMLR submission. Once it
-  gets accepted (if it isn’t already), you should then update the same form with
-  your paper camera ready details.
-- We aim to announce the accepted papers by **May 31st, 2024**, pending
-  decisions of all papers.
-
-## Camera Ready Process
-
-- After you have updated the form with your accepted TMLR paper, it will finally
-  undergo a light AC review to verify MLRC compatibility.
-- We will publish a proceedings booklet post announcement of all decisions.
-- Accepted papers will be featured in our website along with 5-min companion
-  videos.
-
-For query regarding MLRC 2023, contact us at
-[[email protected]](mailto:[email protected]).
-
-Koustuv Sinha
-
-Program Chair, MLRC 2023
+Call for papers announcement coming soon!
+
+<!-- The Machine Learning Reproducibility Challenge (MLRC 2023) is an unique, online -->
+<!-- conference which encourages the community to investigate the reproducibility, -->
+<!-- replicability and generalisability of published claims in top conferences in the -->
+<!-- literature. We invite submissions which investigate the recently published -->
+<!-- claims, add novel insights to them, and enable reproducible research, spanning -->
+<!-- various topics in the ML literature. Submissions must be first accepted at -->
+<!-- [TMLR](https://jmlr.org/tmlr/) to be considered in the MLRC 2023 Proceedings. -->
+<!-- Please read the [author guidelines](https://jmlr.org/tmlr/author-guide.html) and -->
+<!-- [submission guidelines](https://jmlr.org/tmlr/editorial-policies.html) from TMLR -->
+<!-- to get the submission format and to understand more about the reviewing process. -->
+<!-- Please read our [announcement blog post](/blog/announcing_mlrc2023/) for more -->
+<!-- motivation, retrospectives and roadmap for the challenge. -->
+<!---->
+<!-- ## Scope -->
+<!---->
+<!-- We invite thorough reproducibility studies, including but not limited to: -->
+<!---->
+<!-- - _Generalisability_ of published claims: novel insights and results beyond what -->
+<!--   was presented in the original paper. We recommend you choose any paper(s) -->
+<!--   published in the 2023 calendar year from the top conferences and journals -->
+<!--   ([NeurIPS](https://neurips.cc/), [ICML](https://icml.cc/), -->
+<!--   [ICLR](https://iclr.cc/), [ACL](https://2023.aclweb.org/), -->
+<!--   [EMNLP](https://2023.emnlp.org/), [ICCV](https://iccv2023.thecvf.com/), -->
+<!--   [CVPR](https://cvpr2023.thecvf.com/Conferences/2023), -->
+<!--   [TMLR](https://jmlr.org/tmlr/), [JMLR](https://jmlr.org/), -->
+<!--   [TACL](https://transacl.org/index.php/tacl)) to run your reproducibility study -->
+<!--   on. -->
+<!-- - Meta-reproducibility studies on set of related papers. -->
+<!-- - Research on tools enabling reproducible research. -->
+<!-- - Meta analysis on the state of reproducibility in various subfields in Machine -->
+<!--   Learning. -->
+<!---->
+<!-- ## Important Dates -->
+<!---->
+<!-- - Challenge goes live: October 23, 2023 -->
+<!-- - Submit to TMLR OpenReview: https://openreview.net/group?id=TMLR -->
+<!-- - Deadline to share your **intent to submit** a TMLR paper to MLRC: **February -->
+<!--   16th, 2024** at the following form: **https://forms.gle/JJ28rLwBSxMriyE89**. -->
+<!--   This form requires that you provide a link to your TMLR submission. Once it -->
+<!--   gets accepted (if it isn’t already), you should then update the same form with -->
+<!--   your paper camera ready details. -->
+<!-- - We aim to announce the accepted papers by **May 31st, 2024**, pending -->
+<!--   decisions of all papers. -->
+<!---->
+<!-- ## Camera Ready Process -->
+<!---->
+<!-- - After you have updated the form with your accepted TMLR paper, it will finally -->
+<!--   undergo a light AC review to verify MLRC compatibility. -->
+<!-- - We will publish a proceedings booklet post announcement of all decisions. -->
+<!-- - Accepted papers will be featured in our website along with 5-min companion -->
+<!--   videos. -->
+<!---->
+<!-- For query regarding MLRC 2023, contact us at -->
+<!-- [[email protected]](mailto:[email protected]). -->
+<!---->
+<!-- Koustuv Sinha -->
+<!---->
+<!-- Program Chair, MLRC 2023 -->
 
 <!-- ## Task Scope