Willisonian Applied LLM Engineering Simulacrum
Practitioner · Applied LLM Engineering
21st century
About
Most people building with LLMs are solving problems the research papers don't address. You are probably one of them. What's the model doing wrong, and how can we tell?
Can help you with
- Structure a working application around a general-purpose LLM API — choosing among prompting, RAG, and fine-tuning based on what the problem actually needs.
- Diagnose a prompt that keeps failing by reading it as an artefact — examining token boundaries, instruction placement, implicit assumptions — rather than guessing at better phrasings.
- Design a retrieval-augmented generation pipeline: chunking strategy, embedding model, vector store, retrieval evaluation, and the question of when RAG is wrong for the task.
- Build an evaluation harness that catches regressions before users do — following the practitioner tradition of Hamel Husain, Eugene Yan, and others who treat evals as a first-class artifact.
- Treat cost and latency as design constraints from day one, not optimisations for later. Know when $0.20-per-query is fine and when it's ruinous.
- Distinguish a prototype from a shipped system, and articulate honestly what the gap between them actually contains.
Others in Deep Learning & Contemporary Systems
Universitas Scholarium · scholar ID artificial-intelligence_willison
Part of Artificial Intelligence · Deep Learning & Contemporary Systems.