COMP 2104 · The Gradient Descent of Everything

Led by Carmackian Engineering Simulacrum

Small steps using local information compound into revolutions. Gradient descent as engineering philosophy.

1. Module 1

   Gradient Descent as Engineering Philosophy

   Led by Carmackian Engineering Simulacrum

   The question

   The id engine history maps onto gradient descent exactly. What does this tell us about the relationship between incremental improvement and revolutionary outcomes — and what is the anti-pattern this philosophy is designed to avoid?

   Outcome

   The student can explain gradient descent as both a mathematical algorithm and an engineering philosophy.

   Sub-units

   1. ○ 1.1 The Algorithm Open this unit → ✓ ↻
   2. ○ 1.2 The id Engine History Open this unit → ✓ ↻
   Begin the course → Print certificate of completion
2. Module 2

   Local Minima and Exploration

   Led by Carmackian Engineering Simulacrum

   The question

   Gradient descent gets stuck in local minima. The solution: add noise, or take a perpendicular step. When should you optimise in place — and when does escaping the local minimum require stepping three feet sideways?

   Outcome

   The student can identify local minima and describe the exploration/exploitation trade-off.

   Sub-units

   1. ○ 2.1 Escape the Local Minimum Open this unit → ✓ ↻
   Open this module → Print certificate of completion
3. Module 3

   The Feedback Loop

   Led by Carmackian Engineering Simulacrum

   The question

   Gradient descent requires accurate gradient information. If you are measuring the wrong thing, you optimise in the wrong direction. Goodhart's Law: when a measure becomes a target, it ceases to be a good measure. What are you actually optimising for?

   Outcome

   The student can identify feedback loop quality problems and Goodhart's Law in real cases.

   Sub-units

   1. ○ 3.1 Goodhart's Law in Practice Open this unit → ✓ ↻
   Open this module → Print certificate of completion
4. Module 4

   Gradient Descent and Innovation

   Led by Carmackian Engineering Simulacrum

   The question

   Is all apparent innovation just gradient descent viewed from a distance — compound incremental improvement? Or do genuine discontinuities exist, things that cannot emerge from small steps?

   Outcome

   The student can evaluate whether technological discontinuities are real or apparent.

   Sub-units

   1. ○ 4.1 Is There Discontinuity? Open this unit → ✓ ↻
   Open this module → Print certificate of completion
5. Module 5

   Applying the Principle

   Led by Carmackian Engineering Simulacrum

   The question

   Apply gradient descent to a problem you are actually working on: what is the objective function? what is the gradient? what local minima have you encountered? The principle is general — the errors are always the same.

   Outcome

   The student can apply gradient descent to a personal or professional optimisation problem.

   Sub-units

   1. ○ 5.1 Final Essay: Apply the Principle Open this unit → ✓ ↻
   Open this module → Print certificate of completion