Skip to content

Chapter 2.13 — Uncomfortable truths about LLMs

This block summarizes, deliberately uncomfortably, what an LLM is not. It serves as a reference for architecture reviews, onboardings and any conversation in which someone says "the model takes care of that". Each item briefly revisits points discussed throughout Part 2.

Use it as an operational reminder, not as a manifesto.

  • An LLM is not a source of truth. Use grounding, retrieval, a database, a knowledge graph and validation.
  • An LLM should not be an authorization mechanism. Use IAM, RBAC/ABAC, OAuth, policy-as-code.
  • An LLM does not replace critical deterministic rules. Use policy-as-code and validation.
  • Structured output improves integration, but does not eliminate validation. Form without correct content is still invalid.
  • RAG reduces the risk of hallucination, but does not guarantee truth.
  • Fine-tuning is not a dynamic knowledge base.
  • Pure determinism does not exist, even with temperature 0.
  • Changing the model version is a production change, with regression potential, even if the name is the same.

When any of these points clashes with a vendor's or a framework's claim, this book's editorial stance is: believe the list above until proven otherwise, measured in production.