Eurostar has addressed security vulnerabilities in its customer-facing AI chatbot after it was alerted to the issues by cybersecurity firm Pen Test Partners.
Like many large transport operators, Eurostar runs a chatbot on its website to handle customer contact and deflect simple queries from human agents. Unlike older menu-driven bots, Eurostar’s bot accepts free-text input and responds flexibly, indicating that a large language model (LLM) sits behind the interface.
From a customer’s perspective, the interaction with the chatbot feels conversational. Behind the scenes, the chatbot is driven by a REST API that sends the entire conversation history to the backend on every turn. Each message is labelled with metadata and sometimes a cryptographic signature indicating it passed content checks.
On paper, this is a sensible design. Guardrails are used to filter disallowed content before it reaches the model, responses are checked before being shown to the user and messages are tagged with unique identifiers.
But the server-side enforcement and binding were weak, opening the way for an attacker to exfiltrate prompts, steer answers and run scripts in the chat window.
When using the site, Ross Donald, Head of Core Pen Test at Pen Test Partners, conducted a penetration test under Eurostar’s vulnerability disclosure program and uncovered four security flaws in the AI chatbot’s implementation. All of them were rooted in familiar web and API security weaknesses that were amplified by the presence of an LLM at the center of the experience. Controls that existed in the user interface (UI) or architecture were not consistently enforced server-side.
The assessment identified four related weaknesses:
- Guardrail bypass
- Information disclosure via prompt injection
- HTML injection leading to self-XSS
- Conversation and message IDs not being properly verified
The security firm confirmed these were fixed before disclosing them publicly.
Guardrails that Only Look at the Latest Message
The immediate risk in the way the chatbot was set up was the fact that the entire conversation history was sent to the backend API with each response, but only the latest message’s signature was ever checked, Donald wrote in a blog post.
“As long as the most recent message looked harmless and passed the guardrail check, any earlier messages in the history could be altered client-side and would be fed straight into the model as trusted context.”
This would allow an attacker to submit a benign message to pass the guardrail check and get a valid signature, while altering earlier messages client-side to include a malicious prompt, which would be accepted without checking. Donald wrote:
“Using this pattern, I could first bypass the guardrails and then use prompt injection to get the model to reveal information it should not disclose, such as the underlying model name and system prompt.
“At that point I was no longer constrained by the guardrail layer; I was directly up against the model and its internal instructions.”
Prompt Injection and Information Disclosure
“Prompt injection in this case did not expose other users’ data,” Donald wrote. But by exploiting the guardrail weakness, a malicious actor would be able to inject prompts to force the chatbot to reveal its system instructions and underlying model details.
Leaking system prompts reveals how the AI chatbot is structured, how it decides what to say and how it formats responses. That kind of information could be used in future attacks to steal customers’ personal data, especially particularly as the use and functionality of chatbots evolves to handle bookings, loyalty accounts or disruption management.
While Eurostar’s chatbot only had limited functionality at the time, “the bigger risk is what happens once the chatbot is allowed to touch personal data or account details,” Donald wrote.
HTML Injection and Self-XSS in the AI Chatbot Window
The AI chatbot was instructed to return HTML-formatted responses, including links to help articles. Those responses were returned directly in the browser. Once prompt injection was possible, Donald said it was easy to convince the LLM to return arbitrary HTML instead of a normal support link.
In testing, this was limited to harmless examples, but “in a real attack the same primitive could be used to return malicious JavaScript or phishing links inside what looks like a legitimate Eurostar answer,” Donald wrote.
Weak Validation of Conversation and Message IDs
Each conversation and message used UUIDs, but the backend did not properly validate them, accepting arbitrary values such as simple strings without challenge.
“Combined with the weak validation of conversation and message IDs, there is a clear path to a more serious stored or shared XSS where one user’s injected payload is replayed into another user’s chat,” Donald wrote.
The test didn’t run this scenario end-to-end but the lack of validation is a design flaw that should be avoided in customer-facing systems.
Disclosure Friction Is a CX Problem Too
The technical findings are only part of the story. The disclosure process itself highlights an often-overlooked dimension of customer and partner experience: how organisations handle good-faith reports.
Despite Eurostar publishing a vulnerability disclosure program, Donald initially submitted a disclosure of the findings in June, which went unanswered for weeks, as did follow-ups. After a colleague escalated via LinkedIn, Eurostar initially said there was no record of the disclosure, as the train operator had outsourced its VDP after the initial disclosure and launched a new page with a disclosure form. Donald wrote:
“It raises the question of how many disclosures were lost during this process.”
At one point, Eurostar even suggested the researchers were attempting blackmail, despite no threats being made and no response having been received to the original report.
“To say we were surprised and confused by this has to be a huge understatement – we had disclosed a vulnerability in good faith, were ignored, so escalated via LinkedIn private message. I think the definition of blackmail requires a threat to be made, and there was of course no threat.”
“We still don’t know if it was being investigated for a while before that, if it was tracked, how they fixed it, or if they even fully fixed every issue!” Donald added.
The incident is a reminder that trust is shaped not only by frontline interactions, but also by how organizations respond to external signals of risk.
A customer who clicks a chatbot link expects safety by default. If that trust is broken, confidence in the entire digital service erodes, not just the chatbot.
How to Secure AI Chatbots Using Web Security Practices
Donald emphasized the importance of securing AI chatbots. After all, this is far from the first time flaws in customer-facing AI chatbots have come to light.
Earlier this year, researchers tricked Yellow.ai’s chatbot into generating malicious HTML and JavaScript, a reflected XSS issue that could have enabled session cookie theft and account hijacking before it was patched. Similarly, security testers showed that Snapchat’s My AI could be manipulated into sharing harmful content by reframing requests as storytelling, raising concerns about what seemingly benign conversation bots might reveal under creative prompting. Other incidents, such as a small prompt that coaxed Lenovo’s virtual agent into leaking live session cookies, illustrate how easily chatbots can be weaponised without rigorous safeguards.
The good news is that “[t]he fixes for this kind of chatbot are not exotic,” Donald said. In practice, it’s about applying the same web and API security basics you already use, and doing so consistently from build through to day-to-day operation.
It starts with treating prompts and guardrails as real security controls, not just user experience or content decisions, Donald wrote.
“Separate instructions from data so that anything coming from users, websites, or documents is always treated as untrusted content, not extra system prompt. Apply least privilege here as well. Only give the model the tools, data, and actions it genuinely needs for the use case.”
Input and output handling is just as critical. Every input that reaches the model should be validated and sanitized, including IDs and external content. On the way out, don’t render model responses directly as HTML. As Donald noted, “Those HTML snippets were rendered directly in the chat window with no sanitization,” which is an avoidable risk.
Guard decisions should be enforced on the server, and identifiers must be properly validated. “The issues we found around guardrails and IDs are a design problem as much as an implementation one,” Donald wrote.
AI chatbot security has to be an ongoing process. Log and monitor interactions, watch for unusual patterns, have a clear incident response plan and be ready to switch features off if needed. The bottom line is simple: “Old web and API weaknesses still apply even when an LLM is in the loop.”
