AI Engineering Complete Bootcamp — Large Language Models and Transformers

Led by Geoffrey Hinton Simulacrum

The transformer architecture, GPT and BERT model families, embeddings, question answering, and fine-tuning pre-trained models for specific tasks.

1. Module 1

   Transformer Architecture and GPT

   Led by Geoffrey Hinton Simulacrum

   The question

   The problem with RNNs · the attention mechanism · the transformer architecture (input embeddings, multi-headed attention, feed-forward layers, masked attention, output prediction) · what GPT means (Generative Pre-trained Transformer) · the developmen...

   Outcome

   Demonstrates competence in transformer architecture and gpt.

   Sub-units

   1. ○ 1.1 Transformer Architecture and GPT Open this unit → ✓ ↻
   Begin the course → Print certificate of completion
2. Module 2

   BERT, XLNet and Fine-Tuning

   Led by Geoffrey Hinton Simulacrum

   The question

   GPT vs BERT (generative vs bidirectional) · BERT architecture (bidirectional encoder, masked language modelling, next sentence prediction) · BERT embeddings · building a question-answering bot with BERT · BERT variants (RoBERTa, DistilBERT) · XLNet (...

   Outcome

   Demonstrates competence in bert, xlnet and fine-tuning.

   Sub-units

   1. ○ 2.2 BERT, XLNet and Fine-Tuning Open this unit → ✓ ↻
   Open this module → Print certificate of completion