Edition 1 · 02 / 08
The Prompt Is Not the Security Boundary
Agents turn an information question into a permissions question. What can they reach, and what can they do?
Telling an AI assistant not to disclose confidential information is useful guidance. It is not a substitute for controlling the systems, credentials and information available to it.
That distinction becomes critical when the assistant can use tools. A system that only returns text has a different exposure from one that can read repositories, query a database, modify files or deploy software. Both may look like a chat window to the person using them.
From prompts to capabilities
A traditional interaction is relatively easy to describe: a person sends a prompt and receives a response. An agent can also retrieve context and act through connected services. Model Context Protocol, or MCP, standardises connections to tools and information sources. It does not remove the need to assess the permissions and operators behind those connections.
For a customer, the delivery chain can extend from the supplier to an agent platform, a model provider, a tool server and another external service. Different parts may have different identities, retention arrangements and approval controls.
The relevant inventory therefore includes capabilities as well as vendors. Can this identity read customer tickets? Can it change repository content? Can it execute code or reach production? Can information move from one connected system to another?
Why instructions are insufficient
The NCSC explains that current language models do not enforce a robust separation between instructions and untrusted content within a prompt. A document or tool response can contain text that attempts to redirect the model.
The practical implication is to design for instructions failing. Keep sensitive permissions outside the model’s discretion. Give each workflow only the access it needs, separate customer environments and require independent approval for consequential actions.
An instruction can describe a limit. The surrounding system must enforce it.
A human approval step also needs substance. A reviewer should see the proposed action, target and relevant evidence. A vague request to approve “the next step” gives little basis for a sound decision.
Ask for evidence at the boundary
Supplier assurance should examine scoped, revocable credentials; restrictions on network destinations; separation of read and write access; and records of consequential tool calls. It should also establish who can disable an agent or connection during an incident.
A useful exercise is to give the supplier an untrusted document containing misleading instructions and ask how the surrounding system limits the possible consequences. This tests containment, rather than merely whether a model ignores one example.
Review the full chain whenever a new tool or broader permission is introduced. Adding a connector can change exposure even if the model itself stays the same.
The customer’s question becomes concrete: which controls would still protect us if the agent followed the wrong instruction?
References
- NCSC: Prompt injection is not SQL injection. Supports the instruction and data boundary concern; the control questions above are practical editorial recommendations.