High

AI features that can be talked into anything

Users trick your AI into leaking data, prompts or running tools it shouldn't.

What it is

When your app feeds user input, documents or web pages to an LLM, a crafted message can override your instructions, reveal your system prompt, or make the model call tools with other users' data.

Why fast-built apps get it

The AI feature is the product, so it ships first. Its tools often run with full database access, and the model is trusted to decide what's allowed.

Before and after the cure

lib/agent.tsVulnerable

const tools = {
  // The model chooses the customerId, so it can ask for anyone's
  getOrders: ({ customerId }) => db.order.findMany({ where: { customerId } }),
};

lib/agent.tsCured

const tools = (user: User) => ({
  // Scoped by the server to the signed-in user; the model can't change it
  getOrders: () => db.order.findMany({ where: { customerId: user.id } }),
});
// Risky actions (refunds, emails) need the user to confirm in the UI