Skip to main content

AI Security

Prompt injection is an authorization problem

Treating injection as a prompt-engineering issue guarantees you will not fix it. The durable controls sit in the tool surface and the permission model.

7 min read

Prompt injection gets discussed as though it were a wording problem — as if a firmer system prompt would settle it. It will not. Once a model reads untrusted content and can also call tools, that content is effectively submitting instructions to a system holding your credentials. The mitigation belongs where every other authorization control belongs: outside the thing being persuaded.

The shape of the risk

An agent that processes supplier invoices reads documents supplied by third parties. An agent that triages a shared mailbox reads mail from anyone who knows the address. An agent that browses reads whatever the page serves it. In each case the attacker controls text the model will read, and the model cannot reliably distinguish instructions from data in that text.

  • Data exfiltration — the injected content asks the agent to summarize prior context into an outbound message.
  • Unauthorized tool use — the content instructs the agent to call a tool the task never required.
  • Privilege escalation by chain — one agent is persuaded to ask another, more privileged agent to act.
  • Silent corruption — the content alters an extracted value rather than the agent’s behaviour, so nothing looks wrong.

Controls that actually hold

Assume the model will be talked into trying something. Design so that trying does not succeed. That means the controls are structural, not textual.

  • Least privilege per agent, per task. Not a shared service account. Not the permissions of the developer who deployed it.
  • A narrow tool surface. If the task does not need an outbound send tool, the runtime should not expose one.
  • Human approval on any egress or irreversible action, until the disagreement rate justifies otherwise.
  • Provenance on retrieved content, so untrusted sources are marked as untrusted throughout the pipeline.
  • Egress allow-lists on destinations, so an exfiltration attempt has nowhere permitted to go.
  • Monitoring on tool-call patterns, because a successful injection usually looks statistically odd before it looks harmful.

Test it before someone else does

Red team the agent with payloads embedded in realistic documents and messages, not with a list of jailbreak strings. Then convert every finding into a standing evaluation, so the fix is regression-tested on every subsequent change. Injection is not a vulnerability you close once; it is a property of the architecture you keep verifying.

Working on something in this area?

Pentagon X takes organizations from AI opportunity identification through production deployment and ongoing optimization.

This is the thinking behind AI Governance & Security control and secure ai at scale.