Large Language Models

Train and Customize Language Models

AI Programming & Machine Learning4.7(65)3h 18m19 lectures1 section

What you'll learn

  • Understand how language models evolved before Transformers
  • Learn the basics of Tokenization
  • Grasp the Transformer architecture and the Multi-Head Attention mechanism
  • Get to know the BERT model and its applications
  • Master Prompt Engineering
  • Tune the Configuration Parameters
  • Use the Hugging Face platform
  • Summarize YouTube content using Transformers
  • Customize models with Fine-Tuning, PEFT and LoRA

About this course

This course explains large language models (LLMs), how they work, and how they can be customized to build smart applications. It starts with an introduction, then the time before the era of Transformers and the basics of Tokenization. After that it explains the Transformer architecture in four parts, including the Multi-Head Attention mechanism, then the different architectural patterns. You then learn Prompts and Prompt Engineering, the BERT model and its applications, and the Configuration Parameters in language models. You also get to know the Hugging Face platform in three parts, then carry out a hands-on project that summarizes YouTube content using Transformers. It ends with Fine-Tuning and PEFT techniques, which stands for Parameter-Efficient Fine-Tuning, and the LoRA technique, which stands for Low-Rank Adaptation. It suits programmers and anyone interested in AI and language processing, and basic programming knowledge is preferred. By the end you will understand how language models work, and be able to use Hugging Face and customize models for your own applications.

Expected outcomes

  • Comprehensive understanding of training and customizing large language models.
  • Ability to use the Hugging Face platform to work with LLMs.
  • Mastery of PEFT and LoRA techniques for efficient model customization.
  • Capability to build smart applications for text analysis and content summarization.

Learner reviews

These reviews are from learners who bought the content and completed at least 50% of it — there are no reviews from anyone who hasn't gone through the material.
4.7
★★★★★
65 reviews
5★
48
4★
13
3★
4
2★
0
1★
0
  • ★★★☆☆Tamer F.·منذ 5 شهرشاهد 100٪

    الشرح جيد ولكن يحتاج الى توسع اكثر حيث ان الكورس مختصر وأيضا يحتاج الى تدريبات عملية أكثر

تم عرض كل التقييمات (1)

Course content

1

Large Language Models LLMs

19 lectures
  1. Course IntroductionGives an overview of the large language models course: from RNNs, tokenization, the Transformer architecture and its variants and prompt engineering, to Hugging Face, a YouTube summarization project, then fine-tuning, PEFT and LoRA.3:51
  2. Before the Era of TransformersSets the stage for Transformers by explaining sequence-to-sequence tasks such as translation and summarization, how an RNN works step by step with a hidden state, and its flaws: slowness on long sequences and forgetting early information.7:02
  3. Basics of TokenizationExplains tokenization: splitting text into tokens and mapping each token to an ID from a vocabulary the tokenizer builds, then its types at word, character and subword level, including BPE and WordPiece.4:34
  4. Transformers Architecture - Part OneStarts explaining the Transformer architecture with the Encoder: representing words as vectors through input embedding with a model dimension of 512, then positional encoding with sine and cosine equations to add word order.8:02
  5. Transformers Architecture - Part TwoExplains self-attention inside the Encoder: copying the matrix into Query, Key and Value, multiplying Q by transposed K, scaling by the square root, then softmax, to capture how each word relates to its neighbors.5:07
  6. Transformers Architecture - Part Three: Multi-Head Attention MechanismExplains multi-head attention: multiplying Q, K and V by weight matrices and splitting them into several heads, each focusing on a different pattern of relations, then merging them, with Add and Norm layers and the feed-forward network to complete the Encoder.4:54
  7. Transformers Architecture - Part FourExplains the Transformer Decoder: masked multi-head attention that hides future words, then cross attention taking Key and Value from the Encoder, followed by training with a loss and prediction word by word.9:43
  8. Different Architectural Patterns of TransformersExplains the three Transformer architecture types: encoder only, decoder only and encoder decoder, how each is pre-trained, the tasks each suits such as classification, text generation and translation, and example models.10:02
  9. Prompts and Prompt EngineeringExplains prompt, inference, completion and context window, then improving results by adding examples inside the prompt through in-context learning, one-shot and few-shot, and the limits that lead to fine-tuning.6:45
  10. BERT Model and Its ApplicationsExplains BERT, which uses only the Encoder: its architecture and WordPiece tokenizer, how it differs from GPT, and its training through masked language modeling and next sentence prediction, then classification.17:37
  11. Configuration Parameters in LLMsExplains the configuration parameters that control generation at inference time: max tokens, greedy decoding, random sampling, top-k and top-p, and temperature and its effect on the probability distribution.7:52
  12. Hugging Face Platform - Part OneStarts hands-on work on Hugging Face and Google Colab using the transformers pipeline: generating text with GPT-2 and classifying text to check whether a sentence is grammatically correct.15:17
  13. Hugging Face Platform - Part TwoApplies sequence-to-sequence tasks on Hugging Face with pipelines: translating text between languages, summarizing text while tuning max length, min length and do sample, and generating questions from a text.13:41
  14. Hugging Face Platform - Part ThreeShows an alternative to pipelines: loading the model and tokenizer separately, running tokenization and inference manually, decoding while skipping special tokens, and building a function that generates the summary directly.15:15
  15. Practical Project: Summarizing YouTube Content Using Transformers - Part OneStarts the YouTube summarization project: fetching the transcript from the video link through its video ID in Google Colab, and merging the text segments into one variable before feeding it to the summarization model.20:16
  16. Practical Project: Summarizing YouTube Content Using Transformers - Part TwoCompletes the YouTube summarization project: building a function that summarizes the transcript by splitting it into character chunks, passing them through a summarization pipeline and combining the results, then running the whole project on a video link.15:41
  17. Introduction to Fine-TuningIntroduces fine-tuning to specialize a pre-trained model for a specific task, why examples inside the prompt are not enough, how training works on prompt and response pairs, and the steps from preparing the dataset to evaluation.15:17
  18. PEFT Techniques (Parameter-Efficient Fine-Tuning)Explains PEFT techniques that solve the cost of full fine-tuning and the forgetting of old knowledge by training only a small number of parameters, and their types: selective, re-parameterization, additive and soft prompts.7:42
  19. LoRA Technique (Low-Rank Adaptation)Explains LoRA: freezing the original weights and training two small matrices sized by the rank, multiplying them and adding the result to the weights, with benefits such as swapping tasks and how to choose the rank value.9:30

Instructor

Eng. Ziad Ahmed

Eng. Ziad Ahmed

Data scientist and engineer specializing in large language models. Expert in natural language processing, automation, and data analysis.
19.8K views3 courses

Related courses