Chapter 7.1 — Environments and CI/CD¶
🎯 Objective¶
Structure environments and AI-specific CI/CD pipelines. In traditional systems, an "environment" is a copy of the application. In AI, an environment also involves the model, prompts, RAG indexes, tools, policy bundles, memory and data — each with its own lifecycle.
🧠 Recommended environments¶
| Environment | What it is for | Cautions |
|---|---|---|
| Local | Individual development | Local models (Ollama, llama.cpp) to avoid cost and PII at a provider |
| Dev | Shared by the team | Synthetic RAG index; redacted PII data; low budgets |
| Staging | A close mirror of production | Same model as production; staging index; redacted production data |
| Security sandbox | Isolated execution (generated code, experimental tools) | Network blocked by default; minimal egress allowlist |
| Shadow mode | A candidate version receives real traffic without effect | Only for components without side effects; measures the difference vs production |
| Production | Real traffic | SLO, monitoring, budgets, audit |
🧠 AI-specific CI/CD¶
- Model CI/CD. Training/fine-tuning pipeline, eval gate (golden + adversarial + regression), promotion in the model registry with approval.
- Prompt CI/CD. Versioning (semver or alias), regression tests with a golden set, automated eval in the PR.
- Tool CI/CD. Versioned schema, contract tests, security tests (injection in the arguments), compatibility tests between versions.
- Agent CI/CD. End-to-end eval harness on golden + adversarial cases; trajectory tests; budgets in eval.
- MCP server CI/CD. Compatibility tests between versions; capabilities review at each upgrade; artifact signing.
- RAG index CI/CD. Recall and groundedness eval before promoting a new index; dual-read support in migrations.
- Policy CI/CD.
opa test(or equivalent) on every PR; signed bundles.
📌 Principles¶
- The eval gate is non-optional. Changes that affect behavior go through eval; without it, regression enters silently.
- Pinning. Models, serving frameworks, tokenizers, embedding models and container digests are pinned.
- Promotion has an owner. An explicit approver at each environment rung.