- Instructor: James Thorne
- Every Monday and Wednesday: 10:30am — 12:00pm
- Seoul Building 1 International Seminar Room (for students in Hongneung campus) + Zoom (for others)
- Zoom link to be provided on KLMS
- Resources: Students should use their own GPU or use the departmental VESSL cluster. If not available, cloud-based services such as Google Colab can be used.
- Prerequisites: It is recommended to take a class such as AI501 machine learning, AI502 deep learning, or AI504 programming for AI.
- Follow-on: After completion of the course, topics such as AI620 Bias and ethics in NLP (Spring Semester 2024) and AI705 Language Models (Fall and Spring 2024) are recommended
- 4x Problem-based Assignments (90%) Assignments must be completed individually
- Recurrent Neural Networks
- Transformers
- Knowledge Intensive / Open Domain NLP
- Modular / Efficient NLP
- Attendance + In-lab or In-lecture Quiz (10%)
Late days: 4 free late days will be provided for the student to use as they wish, shared between all assignments. After late days are exhausted, 10% penalty per day per assignment will be awarded.
| Week | Date | Monday (Lecture) | Wednesday (Student-Led Team Lab) | Note |
|---|---|---|---|---|
| 1 | 8/28 | Introduction and Tasks | PyTorch Practice | |
| 2 | 9/4 | Word Embeddings | Building Word2Vec, GloVe, PMI Matrix Factorization | |
| 3 | 9/11 | Recurrent Neural Networks for Classification | RNNs / Natural Language Inference | |
| 4 | 9/18 | Recurrent Neural Networks for Generation | Machine Translation | Assignment 1 Due |
| 5 | 9/25 | Attention and Transformer | NLI, Classification with transformer (BERT) | Chuseok |
| 6 | 10/2 | No Class (Chuseok) | Large-Scale Language Modelling / Sequence Generation with Transformers (HuggingFace) | Chuseok |
| 7 | 10/9 | No Class (Hangul Day) | Large-Scale Language Modelling / Pre-training | Hangul Day |
| 8 | 10/16 | No Class | No Class | Assignment 2 Due, Midterm Week |
| 9 | 10/23 | Question Answering | SQuAD, Fusion In Decoder | |
| 10 | 10/30 | Information Retrieval | DPR, GENRE, AutoRegressive Search Engines | |
| 11 | 11/6 | Tagging and Structured Prediction | HMM, CRF, ASP, QaNER | |
| 12 | 11/13 | Open-Domain NLP | FEVER, KILT, HotpotQA | Assignment 3 Due |
| 13 | 11/20 | Large Language Models, Chain of Thought, Instruction Tuning | Zero Shot Chain of Thought | |
| 14 | 11/27 | Modular Learning, Parameter Efficient Fine-Tuning and Prompt Tuning | PEFT, P-Tuning, Adaptors, Recurrent Transformer Model | |
| 15 | 12/4 | Diffusion Language Models (TBD) | TBD | |
| 16 | 12/11 | No Class | No Class | Assignment 4 Due, Finals Week |