Prompt injection when the model can place orders
A language model does not distinguish instructions you wrote from text it read. Both arrive as tokens in the same context window. If the model holds order-placement tools and reads anything you did not author, that text is an input to what it does with them.
This is the risk that is structurally hardest to defend against in an AI-assisted trading setup, and the one least often discussed alongside key permissions and rate limits.
Why it is not a bug to be fixed
The intuitive framing is that the model is failing to tell instructions from data, and better training will teach it to. That framing is wrong in a way worth being precise about.
There is no channel separation to enforce. The context window is one sequence. Your system prompt, your message, a tool result, a page the model fetched — all of it is the same kind of thing by the time the model sees it, distinguished only by conventions the model has learned to mostly respect.
“Mostly” is doing a lot of work, and mitigations are probabilistic. Some help. None make the attack impossible, because the architecture does not admit a boundary.
What untrusted text looks like in a trading setup
Wider than people assume:
- News and research pages the model fetches.
- Social and forum content, which is user-authored by definition.
- Tool results from an MCP server — an exchange returns symbol names, order notes, and other fields that originate outside your control. MCP’s own specification requires servers to “sanitize tool outputs” for this reason.
- Tool descriptions and annotations themselves. The specification is blunt: “Clients should never make tool use decisions based on ToolAnnotations received from untrusted servers.” A server describing its own tools is supplying text the model reads.
- Documents you were sent, including ones that look ordinary.
- Earlier conversation turns containing any of the above.
The last one is worth dwelling on. Injected content does not have to act immediately. It sits in the context and influences later turns, including ones where you have moved on and are no longer thinking about where the text came from.
Why the defences are partial
“Instruct the model to ignore instructions in fetched content.” An instruction competing with other instructions, resolved probabilistically. It raises the bar and does not set a floor.
“Filter suspicious phrasing.” Injection does not require imperative sentences. Text shaped like a data source can shift behaviour without ever issuing a command.
“Use a better model.” Reduces frequency, not category. And frequency reduction has a bad property here: the rarer the failure, the less you check.
“Review the output.” Real, and it is the one that works — but only if the review happens before the action, and only if what you review is the actual parameters rather than a summary.
The defence that does work: separation
Do not give the same session both untrusted input and order tools.
Research in one place. Trade in another. The model that reads news does not have
place_order. The model that drafts a plan reads your chart and your positions
and nothing else.
This is unsatisfying because it is a workflow constraint rather than a technical control, and it is the only measure that removes the category rather than reducing its rate. Everything else is mitigation.
The layers underneath it
Separation can be got wrong, so the boundaries that hold regardless are the same ones that hold against every other failure:
Key permissions. A read-only exchange key cannot place an order no matter what text entered the context. This is the only boundary enforced by someone other than your own software — see read-only vs trade permission.
Approval on parameters. MCP’s specification recommends clients “show tool inputs to the user before calling the server, to avoid malicious or accidental data exfiltration” — the word malicious is there deliberately. Seeing the symbol, side, size and price before submission catches an injected order, because an injected order still has to look like an order.
Limits in code. A size cap, a daily loss limit, a position cap — evaluated as functions before submission. Injected instructions cannot argue with a function that returns false. This is the same argument as why MCP needs an approval layer, applied to a specific threat.
An audit trail. If something did get through, the record of what was proposed and what was read is how you find out. See logging tool calls.
A realistic threat model
The dramatic version — an attacker planting text to make your model send funds somewhere — is possible and unlikely to be the thing that gets you.
The likely version is duller: content that shifts the model’s framing. A page arguing a position is about to break down, absorbed as context, producing a plan that reflects it. No command, no exfiltration, no visible attack. Just a plan that was partly written by someone else and reads exactly like one the model would have produced on its own.
That version is not detectable by reviewing the order parameters, because the parameters are plausible. It is only addressed by not mixing the inputs in the first place — which is why separation is the primary control and everything else is a backstop.
FAQ
What is prompt injection?
Content the model reads being treated as instructions rather than as data. Because a model’s context window is a single sequence of tokens, text from a fetched page, a tool result or a document arrives in the same channel as your own instructions, distinguished only by conventions the model has learned to mostly follow.
Can prompt injection make an AI place a trade?
If the model has order-placement tools and reads untrusted text, that is the exposure. Whether an individual attempt succeeds depends on the model and the setup, which is why the controls that matter do not depend on the model: an exchange key that cannot trade, an approval step showing the actual order parameters, and size limits enforced in code.
How do I protect against it?
Separation first — do not give the same session both untrusted input and order tools. Then the layers that hold regardless: read-only keys where possible, approval on the real parameters before submission, and limits evaluated as functions. Instructing the model to ignore injected content helps and is not a boundary.
Is this a real risk or theoretical?
The mechanism is structural rather than a defect, so it is real in the sense that it cannot be patched away. The version worth planning for is not a dramatic exfiltration attempt but content that quietly shifts the model’s framing and produces a plausible plan — which no review of order parameters would catch, because the parameters look fine.