This repository develops a training curriculum for Generative AI. Rather than starting with the technical aspects of training and fine-tuning Generative AI models, the curriculum will first teach students the skills and knowledge that they need to start deploying Generative AI models immediately. Once students master the skills to deploy ethical and responsible Generative AI solutions, they will then focus on the more technical aspects of prompt-engineering, fine-tuning, and training of Generative AI models from scratch. Below is the current outline of the curriculum.
Course covering practical aspects of deploying, optimizing, and monitoring Generative AI models. The course is divided into three modules: Deployment, Model Optimization, and Monitoring and Maintenance Deployments.
Covers various strategies for deploying Generative AI models starting from local deployment of Generative AI models on a laptop or workstation, followed by on-premise server-based deployments, then edge deployments, before finishing with cloud-based deployments. Cover the pros and cons of each strategy and the factors to consider when choosing a deployment strategy.
Cover techniques for optimizing Generative AI models for deployment, such as model pruning, quantization, and distillation. Cover the trade-offs between model size, speed, and performance.
Cover the importance of monitoring the performance of deployed models and updating them as needed. Discuss potential issues that might arise during deployment and how to troubleshoot them.
- O'Reilly: Generative AI with LangChain
- DeepLearning AI: LangChain for LLM App Development
- DeepLearning AI: LangChain: Chat with your Data
- DeepLearning AI: LangChain: Functions, Tools, and Agents
- DeepLearning AI: LLM Apps with JavaScript
- Latent Space Podcast: LangChain
- DeepLearning AI: Building and Evaluating Advanced RAG Apps
- DeepLearning AI: Agentic RAG Apps
- DeepLearning AI: RAG Apps with JavaScript
- Deep Learning AI: Multi-Agent Systems
- Latent Space Podcast: Llama Index
- DeepLearning AI: Vector Databases: Embedding to Applications
- DeepLearning AIL Embedding Models from Architecture to Implementation
- DeepLearning AI: Knowledge Graphs for RAG
- DeepLearning AI: Advanced AI Retrieval
- DeepLearning AI: Building Applications with Vector databases
- DeepLearning AI: Open Source Models with HuggingFace
- DeepLearning AI: Building Multi-modal RAG Apps
- DeepLearning AI: Federated Learning
- DeepLearning AI: Function Calling and Data Extraction
- DeepLearning AI: Generative AI for Software Development
Discuss the ethical considerations of deploying LLMs, including issues of bias, fairness, transparency, and accountability. Discuss real-world examples where these issues have arisen and how they were addressed.
Teach students about best practices for responsible AI, including techniques for auditing and mitigating bias in LLMs, transparency and explainability techniques, and guidelines for human oversight.
Provide an overview of the legal and regulatory landscape for AI and LLMs. Discuss issues such as data privacy, intellectual property rights, and liability.
This section would cover the various metrics used to evaluate the performance of LLMs. This could include Perplexity, BLEU, ROUGE, and others. Each metric should be explained in detail, including what it measures and its strengths and weaknesses.
Teach students how to evaluate the performance of LLMs using the discussed metrics. This should include practical exercises where students can apply these metrics to evaluate LLMs on various tasks.
Provide students with practical exercises where they can practice evaluating LLMs. We will use real-world examples and datasets provided by our partners for these exercises to give students practical experience.
- DeepLearning AI: LLMs for Semantic Search
- DeepLearning AI: Preprocessing Unstructured Data for LLM Apps
- Latent Space Podcast: Benchmarks 101
- https://github.com/openai/evals
- https://github.com/EleutherAI/lm-evaluation-harness
Discuss the concept of prompt optimization and its importance in effectively using LLMs. Explain how carefully optimized prompts can guide the model's responses and improve its performance.
Teach students various techniques for designing effective prompts. This would include methods for crafting initial prompts, techniques for iterative refinement, and strategies for testing and evaluating prompts.
- O'Reilly: Prompt Engineering for Generative AI (GitHub, Udemy)
- DeepLearning AI: Prompt Engineering with Llama 2/3
- DeepLearning AI: ChatGPT Prompt Engineering for Developers
- DeepLearning AI: Building systems with ChatGPT
- DeepLearning AI: Getting Started with Mistral
- DeepLearning AI: Prompt Engineering for Vision Models
- Udemy: Prompt Engineering for AI
- Open AI: Prompt Engineering Guide
- https://mitchellh.com/writing/prompt-engineering-vs-blind-prompting
- Anthropic Prompt Engineering Courses
Prompt-tuning introduces additional parameters into prompts and then optimizes those parameters using supervised samples.
Discuss the numerous trade-offs involved when deciding which of prompt engineering or prompt tuning (or both!) is the best approach for tuning an LLM.
Discuss the concept of fine-tuning and its importance in effectively using LLMs. Explain how fine-tuning adjusts the pre-trained models to perform specific tasks.
- Mastering LLMs: End-to-end Fine-tuning and Deployment
- Latent Space Podcast: The End of Fine-Tuning
- Latent Space Podcast: Axolotl
- https://github.com/meta-llama/llama-recipes
Cover approaches such as Transfer Learning and Knowledge Distillation.
Cover approaches such as Prefix Tuning and Low Rank Adaptation (LoRA).
Cover the basic ideas of Instruction Tuning as well as extensions such as the popular Reinforcement Learning through Human Feedback (RLHF).
Cover approaches to fine-tuning, such as Direct Preference Optimization (DPO), where the goal of the fine-tuning is to improve the alignment of LLMs with human preferences.
Overview of LLMs and their applications. Understanding the Transformer Model Architecture. The Original LLM Scaling Laws. Discussion on whether to build or use pre-trained LLM models.
- DeepLearning AI: How Diffusion Models Work
- DeepLearning AI: Generative AI with LLMs
- Building LLMs from Scratch (GitHub)
- Building GPT-2 from Scratch
Suggested organization of materials for this course.
- Read* Chapters 1 and 2 on implementing the data loading pipeline (https://manning.com/books/build-a-large-language-model-from-scratch… & https://github.com/rasbt/LLMs-from-scratch…).
- Watch Karpathy's video on training a BPE tokenizer from scratch (https://youtube.com/watch?v=zduSFxRajkE…).
- Read Chapters 3 and 4 on implementing the model architecture.
- Watch Karpathy's video on pretraining the LLM.
- Read Chapter 5 on pretraining the LLM and then loading pretrained weights.
- Read Appendix E on adding additional bells and whistles to the training loop.
- Read Chapters 6 and 7 on finetuning the LLM.
- Read Appendix E on parameter-efficient finetuning with LoRA.
- Check out Karpathy's repo on coding the LLM in C code (https://github.com/karpathy/llm.c).
- Check out LitGPT to see how multi-GPU training is implemented and how different LLM architectures compare (https://github.com/Lightning-AI/litgpt…).
- Build something cool and share it with the world.
(Read = read, run the code, and attempt the exercises 😊)
Understanding the hardware requirements for training LLMs. The role of memory and compute efficiency in LLM training. Techniques for parallelization, including gradient accumulation, asynchronous stochastic gradient descent optimization, and micro-batching.
The importance of dataset diversity and quality in LLM training. Techniques for dataset collection, including crawling public data, online publication or book repositories, code data from GitHub, Wikipedia, news, social media conversations, etc. Dataset pre-processing. Tokenization.
Steps involved in pre-training an LLM. Techniques for model evaluation. Understanding bias and toxicity in LLMs. Instruction tuning.