Prompt injection

Definition

An attack or failure mode where instructions encountered by an AI system cause it to ignore, reinterpret or conflict with its intended instructions.

Put simply

The AI reads something that effectively tells it to do something it shouldn't.

What boards and leaders should understand

This becomes much more consequential when AI can both consume untrusted information and take actions.

For example, an agent reading an external webpage, email or document might encounter instructions intended to make it disclose information or misuse a connected system.

The risk is therefore closely linked to the agent’s permissions and authority.

Questions to consider

  • Which of our AI systems can both consume untrusted external information and take consequential actions?