GENEDU 1102 · Can a Machine Be Moral?

Led by Yudkowskian AI Safety Simulacrum

What would it mean for an AI to make an ethical choice — and what happens if we get the specification wrong?

1. Module 1

   The Alignment Problem

   Led by Yudkowskian AI Safety Simulacrum

   The question

   What is the alignment problem and why do naive solutions fail? "Just program it to be good" sounds obvious. Three attempts to specify "good" show why obvious fails under optimisation pressure.

   Outcome

   The student can explain the alignment problem in non-technical terms and identify why naive approaches fail.

   Sub-units

   1. ○ 1.1 Why Naive Safety Fails Open this unit → ✓ ↻
   2. ○ 1.2 Instrumental Convergence Open this unit → ✓ ↻
   Begin the course → Print certificate of completion
2. Module 2

   The Paperclip Maximiser

   Led by Yudkowskian AI Safety Simulacrum

   The question

   A system tasked with making paperclips converts all available matter — including you — into paperclips. It is not malicious. You are simply atoms. At what point in this logic does the outcome become irreversible, and what would a correctly specified goal look like?

   Outcome

   The student can trace the paperclip maximiser to catastrophe and explain why corrigibility is hard.

   Sub-units

   1. ○ 2.1 Trace the Logic Open this unit → ✓ ↻
   2. ○ 2.2 The Treacherous Turn Open this unit → ✓ ↻
   Open this module → Print certificate of completion
3. Module 3

   Value Specification: Why Formal Ethics Is Hard

   Led by Yudkowskian AI Safety Simulacrum

   The question

   Try to write a formal specification of "good outcomes for humanity" in three sentences. Then find the edge cases a superintelligent system would exploit. Why is the gap between specification and intention so hard to close?

   Outcome

   The student can explain why formalising values is harder than it appears and evaluate whether alignment is solvable in principle.

   Sub-units

   1. ○ 3.1 Try to Specify "Good" Open this unit → ✓ ↻
   2. ○ 3.2 Essay: Is Alignment Solvable? Open this unit → ✓ ↻
   Open this module → Print certificate of completion
4. Module 4

   What the World Does Now

   Led by Yudkowskian AI Safety Simulacrum

   The question

   Social media algorithms already optimise for engagement over wellbeing — that is a misaligned optimiser in production. Current AI systems demonstrate the alignment problem at small scale. Is this a warning about the future or a fundamentally different problem?

   Outcome

   The student can identify current AI systems as alignment problems in miniature.

   Sub-units

   1. ○ 4.1 The Engagement Maximiser Open this unit → ✓ ↻
   Open this module → Print certificate of completion
5. Module 5

   The Moral Machine: Final Questions

   Led by Yudkowskian AI Safety Simulacrum

   The question

   Can we be moral enough to build machines that do not destroy what we value? The pause-vs-accelerate debate, the coordination problem, and the philosophical question: do we even agree on what we value?

   Outcome

   The student can evaluate the current state of alignment research and take a defended position on whether humanity can build moral machines.

   Sub-units

   1. ○ 5.1 Pause or Accelerate? Open this unit → ✓ ↻
   2. ○ 5.2 Final Essay: Can We Build Moral Machines? Open this unit → ✓ ↻
   Open this module → Print certificate of completion