update week10

hao-ai-lab · Mar 11, 2024 · bbf9198 · bbf9198
1 parent 776adfe
commit bbf9198
Show file tree

Hide file tree

Showing 3 changed files with 7 additions and 4 deletions.
diff --git a/_modules/week-10.md b/_modules/week-10.md
@@ -6,12 +6,13 @@ status: Active
 
 Mar 11
 : **1**{: .label} 
-  : [Slides]() &#8226; [Recording]() &#8226; [Scribe Notes](#)
+  : [Slides](assets/slides/22_ml-system-2.pdf) &#8226; [Recording]() &#8226; [Scribe Notes](#)
 : *Reading:*
 * [Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning (required)](https://arxiv.org/pdf/2201.12023.pdf)
-* [PGPipe: Easy Scaling with Micro-Batch Pipeline Parallelism (required)](https://arxiv.org/pdf/1811.06965.pdf)
+* [GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism (required)](https://arxiv.org/pdf/1811.06965.pdf)
 * [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism (required)](https://arxiv.org/pdf/1909.08053.pdf)
 * [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (optional)](https://arxiv.org/pdf/2205.14135.pdf)
+* [Efficient Memory Management for Large Language Model Serving with PagedAttention (optional)](https://arxiv.org/pdf/2309.06180.pdf)
 
 
 
@@ -21,9 +22,10 @@ Mar 13
   : [Slides]() &#8226; [Recording](#) &#8226; [Scribe Notes](#)
 : *Reading:* 
 * [Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning (required)](https://arxiv.org/pdf/2201.12023.pdf)
-* [PGPipe: Easy Scaling with Micro-Batch Pipeline Parallelism (required)](https://arxiv.org/pdf/1811.06965.pdf)
+* [GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism (required)](https://arxiv.org/pdf/1811.06965.pdf)
 * [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism (required)](https://arxiv.org/pdf/1909.08053.pdf)
 * [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (optional)](https://arxiv.org/pdf/2205.14135.pdf)
+* [Efficient Memory Management for Large Language Model Serving with PagedAttention (optional)](https://arxiv.org/pdf/2309.06180.pdf)
 
 
 
@@ -33,7 +35,8 @@ Mar 15
   : [Slides](#) &#8226; [Recording]() &#8226; [Scribe Notes](#)
 : *Reading:* 
 * [Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning (required)](https://arxiv.org/pdf/2201.12023.pdf)
-* [PGPipe: Easy Scaling with Micro-Batch Pipeline Parallelism (required)](https://arxiv.org/pdf/1811.06965.pdf)
+* [GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism (required)](https://arxiv.org/pdf/1811.06965.pdf)
 * [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism (required)](https://arxiv.org/pdf/1909.08053.pdf)
 * [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (optional)](https://arxiv.org/pdf/2205.14135.pdf)
+* [Efficient Memory Management for Large Language Model Serving with PagedAttention (optional)](https://arxiv.org/pdf/2309.06180.pdf)
 
diff --git a/assets/slides/21_ml-system-1.pdf b/assets/slides/21_ml-system-1.pdf
diff --git a/assets/slides/22_ml-system-2.pdf b/assets/slides/22_ml-system-2.pdf