The new exposure
A conventional application processes input it treats as data. An AI application processes input that the model may treat as instruction. That single difference produces most of the novel risk.
Content retrieved from a document, a web page, an email or a ticket can contain text directed at the model. If the system grants the model tools with real permissions, untrusted content becomes a potential path to action.
Controls that actually help
Treat all retrieved and user-supplied content as untrusted data, never as instruction. Keep the system prompt and tool definitions outside anything the model ingests at runtime.
Apply least privilege to tools, not just to users. An agent should hold the narrowest possible set of capabilities, and irreversible actions should require explicit human approval regardless of confidence.
Enforce output handling. Model output rendered into a browser, a shell or a database query needs the same escaping and parameterisation discipline as any other untrusted string.
Log every retrieval, model call and tool invocation with enough fidelity to reconstruct a sequence of events after an incident.
Data protection in retrieval
Indexes built without reference to source permissions are a common and serious weakness. Retrieval must evaluate entitlements at query time, and index content should carry the access metadata required to do so.
Where content leaves your boundary to a model provider, the classification, retention and regional requirements applying to that content should be established before the integration is built, not after.
Make it part of the pipeline
These controls degrade quickly if they depend on review discipline. Evaluation for prompt injection resistance, permission enforcement tests and output handling checks belong in CI, alongside dependency and configuration scanning.
AI security is not a separate programme. It is application security extended to cover a component that can be persuaded.
Written by
The EXSTRONIX team
Perspectives drawn from the AI, engineering, finance and operations work we deliver. General in nature and not a substitute for advice specific to your circumstances.