Skip to content

Chapter 9.1 — Core concepts (mini-glossary)

Just as a quick reference; the full glossary is in glossary.md:

  • A2A — an emerging protocol for interoperability between agents; based on Agent Cards and a task lifecycle.
  • Agent harness — the engineering around the model that makes the agent operable, safe, testable and observable.
  • Agent runtime — the layer that executes the agent's loop.
  • Agency — the operational capacity to pursue goals via intermediate decisions.
  • AgentOps — practices for operating agents with tools, state and runtime.
  • AI gateway — a governance layer between the application and model providers: routing, budget, caching, observability, DLP, policy.
  • AWQ — Activation-aware Weight Quantization (quantization that preserves salient channels).
  • Limited autonomy — restricted by scopes, permissions, budgets and approvals.
  • Context engineering — the discipline of selecting, ordering and structuring context.
  • Continuous batching — a scheduling technique that adds new requests to the batch at each step.
  • CycloneDX — OWASP's SBOM format; supports ML-BOM.
  • Eval harness — continuous evaluation infrastructure.
  • GGUF — llama.cpp's file format for quantized models.
  • GPTQ — layer-by-layer post-training quantization with a second-order approximation.
  • GraphRAG — RAG that uses a graph structure for relations.
  • Groundedness — how much the answer is supported by evidence.
  • KV cache — cache of attention keys/values; central in decode.
  • Matryoshka embeddings — embeddings trained so that prefixes are semantically valid (Kusupati et al., 2022).
  • MCP — Model Context Protocol.
  • Memory poisoning — contamination of memory with incorrect/malicious data.
  • Model routing — dynamic model selection.
  • NF4 — 4-bit "NormalFloat", a format used in QLoRA.
  • PagedAttention — an algorithm for managing the KV cache in paged blocks (vLLM).
  • Policy-as-code — codified, testable and versioned policies.
  • Prompt engineering — formulating instructions.
  • Prompt injection — input that tries to change the model's behavior.
  • QLoRA — fine-tuning LoRA adapters over a 4-bit base (NF4).
  • RAG — Retrieval-Augmented Generation.
  • SBOM — Software Bill of Materials.
  • SLSA — Supply-chain Levels for Software Artifacts.
  • Speculative decoding — a fast draft model proposes tokens; the large model verifies in parallel.
  • SPDX — the Linux Foundation's SBOM format; ISO/IEC 5962:2021.
  • Tool registry — a versioned catalog of tools.
  • TTFT — Time To First Token (prefill latency).
  • TPOT — Time Per Output Token (latency per generated token).