Skip to content

Chapter 0.2 — Probabilistic systems in deterministic environments

🎯 Chapter objective

Show that AI systems are probabilistic, but operate inside deterministic environments (APIs, databases, contracts, regulation) — and that this boundary requires explicit architectural decisions.

🧠 Core concept

An ML/LLM model produces outputs that carry uncertainty. A REST API consuming that output expects a response with a clear contract. When these two layers mix without mediation, the classic problem arises:

  • the probabilistic response becomes a deterministic effect in the real world;
  • a semantic error becomes a CRM update, an email sent, a payment triggered.

The architecture must isolate the variance between the two layers with:

  • schema validation;
  • deterministic business rules;
  • human approvals for critical actions;
  • translation layers (parsing, normalization, controlled mapping).

🏗️ How this shows up in production

Layer Nature Example
LLM/ML Probabilistic LLM-generated answer, risk classification
Post-processing Deterministic Schema validation, normalization
Policy Deterministic "If amount > X, require approval"
Execution Deterministic Database update, external API call
Auditing Deterministic Immutable log of the decision

⚖️ Trade-offs

  • More deterministic layers -> more safety, less flexibility.
  • Fewer deterministic layers -> more agility, more residual risk.

🚨 Failure modes

  • Treating an LLM response as truth.
  • Allowing the model output to trigger an irreversible action without mediation.
  • Confusing schema-validated output with correct output (schema validates form, not content).

🛡️ Controls and mitigations

  • Structured outputs + schema validation as the first line.
  • Policy-as-code for critical rules.
  • Human-in-the-loop for high-impact actions.
  • Complete audit trail for investigations.

🔭 How to observe and measure

  • Rate of responses that fail schema validation.
  • Rate of blocked policy violations.
  • Time between model proposal and human approval.

🧪 How to test

  • Contract tests at every boundary.
  • Adversarial tests that force invalid outputs.
  • Policy-as-code tests with positive and negative cases.
  • EX-FUND-02 — an LLM wrapper with structured output + Pydantic validator + a simple approval policy.

📌 Checklist

  • [ ] Does the model output go through schema validation before any action?
  • [ ] Are critical rules in code, not in the prompt?
  • [ ] Do irreversible actions require approval?

📚 References