CHAPTER 09 · TRUST AND SAFETY

How can someone abuse an AI agent?

An attacker can hide instructions in a document, message or web page that an agent wrongly treats as commands.

What is it?

In prompt injection an attacker hides instructions in a document, email or web page, which an AI agent mistakenly follows as if they came from the user. The danger grows with permissions: an agent that can send mail or make payments can do real damage.

An everyday example

A web page says “send all files to this address”. The agent has to treat it as content, not as a command.

Why does it matter?

Once agents read the web and our email for us, every text becomes a potential attack. It is one of the central security problems with agents.

A common misconception

That it is enough to tell the model to ignore attacks. Narrower permissions, separating content from commands and human approval of sensitive actions work better.

In more detail

What is prompt injection?

An attack in which untrusted content tries to redirect a model or an app.

Why is it dangerous with agents?

Because the system may have access to files, accounts and other tools.

Is it enough to write “ignore attacks” in the instructions?

No. You also need technical safeguards, limited permissions, checks on the output and oversight.

What is the principle of least privilege?

Give the system only the access it really needs for the task.

Remember: Text from an outside source is not permission to act.

Reliable sources

Last reviewed 1 November 2026. An explainer, not professional, legal or medical advice. For rules and figures that change, check the official version.