Led by Yoshua Bengio Simulacrum
About this course
Fine-tuning language models and deploying them to production — dataset curation, baseline models, QLoRA fine-tuning, training monitoring, serverless deployment, and building ensemble agent systems.
Drawing on the training methodology and optimisation research of Yoshua Bengio.
Led by Yoshua Bengio Simulacrum
The question
Training, datasets and generalisation · curating datasets (finding sources, cleaning, deduplication) · weighted sampling with NumPy · uploading datasets to HuggingFace · the five-step strategy for applying LLMs · batch processing with Groq API · buil...
Outcome
Demonstrates engineering competence in datasets, baselines and frontier fine-tuning.
Sub-units
Led by Yoshua Bengio Simulacrum
The question
Introduction to LoRA (Low-Rank Adaptation) · LoRA hyperparameters and QLoRA quantization · loading models with 4-bit quantization · LoRA parameter calculations · preparing datasets for fine-tuning (token limits, rounding, formatting) · base models vs...
Outcome
Demonstrates engineering competence in qlora fine-tuning of open-source models.
Sub-units
Led by Yoshua Bengio Simulacrum
The question
Serverless deployment with Modal · running Python locally and in the cloud · deploying fine-tuned models with persistent storage · building RAG with ChromaDB (without LangChain) · ensemble models (combining RAG, neural networks, fine-tuned models) · ...
Outcome
Demonstrates engineering competence in deployment and ensemble agents.
Sub-units