Skip to content

Chapter 3.7 — A prompt is not a policy, a tool description is not security, deprecation is not just metadata

🎯 Objective

Consolidate three anti-patterns that deserve explicit emphasis, since they are a recurring source of incidents in corporate agents.

🔐 3.7.1 — A prompt is not a policy

Writing "do not do X" in the prompt may help, but it does not guarantee behavior. Critical policies must live in:

  • code;
  • authorization;
  • sandbox;
  • validation;
  • runtime.

Examples:

  • Instead of "do not send email to external domains" in the prompt, remove the permission from the credential and block egress at the network level.
  • Instead of "do not access sensitive data" in the prompt, apply RBAC/ABAC in the retriever and mandatory per-tenant filters.

🔧 3.7.2 — A tool description is not security

A tool's description helps the model choose; it does not prevent misuse. Real security requires:

  • a tool registry with permissions;
  • an allowlist;
  • a policy engine;
  • explicit authorization;
  • argument validation;
  • HITL;
  • logs and auditing;
  • rate limits and circuit breakers.

🗑️ 3.7.3 — Deprecation is not just metadata

Marking a tool as deprecated in metadata does not prevent the model from trying to use it. Prompts saying "do not use this tool" also do not guarantee behavior — they are probabilistic instructions in free text. The tool description, likewise, is not a security mechanism: it is discovery metadata.

Real enforcement must happen in deterministic layers, independent of the model:

Layer Responsibility
Tool registry Marks the tool as deprecated, returns a suggested replacement, records owner and window. Visualization.
Discovery / lazy loading Does not expose a deprecated tool to new agents; existing agents receive a warning
Harness / executor Blocks calls to tools with status removed or blocked. Allows with a warning for deprecated.
Policy engine Refuses via Rego/Cedar when an explicit policy requires not deprecated
Credentials / IAM Revokes access to the removed tool's backend
Allowlist in the executor Explicit list of tools allowed for that agent
Audit log Every attempt to use a deprecated tool is recorded
Operational alert A monitor of deprecated usage triggers a migration review

Recommended migration-window flow:

  1. Announce deprecation with a target removal date.
  2. Mark deprecated: true in the registry.
  3. Measure current usage (tool, agent, tenant).
  4. Create a replacement with schema + parity tests.
  5. Communicate to consumers and owners.
  6. Block new agents/tenants via registry + policy.
  7. Move existing consumers in waves; each wave requires a comparative eval.
  8. After cutover, move the status to removed; the block stays in the executor.
  9. Keep the audit trail for a period defined by compliance.

Summary. Metadata helps discovery. Blocking is the responsibility of the execution path. Without it, "deprecated" becomes a suggestion and the model keeps using the old tool.

🧪 3.7.4 — What is not yet consensus in tool lifecycle for agents

This subsection is an invitation to think critically: many practices around tool lifecycle are forming right now. There are emerging patterns, not closed consensus.

Open points:

  • Versioning granularity. Semver for a tool API works, but side effects (e.g., a new validation) can break agents without changing the schema. There is no dominant standard for versioning "behavior" separately from "interface".
  • Schema compatibility with prompt caching. When you change a tool, cached prompts can become stale and produce invalid tool calls. How to invalidate the cache finely is an open problem.
  • Shadow mode for tools. Conceptually useful (running the new tool in parallel without effect), but tools with real side effects have no true "shadow" without a full sandbox, which is expensive.
  • A2A and versioning between agents. When an agent B is consumed by agent A via A2A, breaking changes in B propagate. Contract and window mechanisms are still being standardized by the A2A spec itself.
  • Community MCP servers and supply chain. How to handle a tool exposed by a third-party MCP server that changes without notice? Emerging pattern: version pinning + signing + capability review.
  • LLM-assisted migration. There are experiments with LLMs rewriting agent code for a new tool version. Useful in prototyping, dangerous in production without eval.
  • Sunset vs hard removal. When a tool is risky (e.g., a deletion with a side effect), is it enough to mark it deprecated or does it require immediate removal? The policy depends on the risk, not the calendar.

This book's editorial position:

  • Version interface AND behavior.
  • Treat the cache as a versioned artifact; invalidate it when changing a tool.
  • Treat MCP servers/A2A peers as supply chain dependencies.
  • Consider shadow mode useful only when there is a sandbox; otherwise, it is an illusion.
  • Do not delegate critical tool migration to an LLM without eval and human review.
  • Document each window as an ADR.

📌 Checklist

  • [ ] Is there a clear separation between probabilistic instruction (prompt) and deterministic control (code)?
  • [ ] Does every sensitive tool have an allowlist, policy and authorization?
  • [ ] Does deprecation have deterministic steps, not just metadata?
  • [ ] Is there a documented window, usage metrics and progressive blocking?
  • [ ] Is there a defined rollback?
  • [ ] Do the MCP servers and A2A peers used have version pinning and capability review?